-- Logs begin at Tue 2022-07-12 17:26:15 UTC, end at Tue 2022-07-12 18:49:32 UTC. -- Jul 12 18:04:34.874055 dv-cinder-844881-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 18:04:34.883180 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 18:04:34 2022] *** Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: nodename: dv-cinder-844881-1-iscsi Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: machine: x86_64 Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: clock source: unix Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: pcre jit disabled Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: detected number of CPU cores: 4 Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: current working directory: / Jul 12 18:04:34.885233 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: detected binary path: /usr/bin/uwsgi-core Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: your processes number limit is 47680 Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: your memory page size is 4096 bytes Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: detected max file descriptor number: 2048 Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: lock engine: pthread robust mutexes Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: thunder lock: enabled Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 18:04:34.885877 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 18:04:34.929677 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: Python main interpreter initialized at 0x560c62bcf9c0 Jul 12 18:04:34.929677 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: python threads support enabled Jul 12 18:04:34.929677 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: your server socket listen backlog is limited to 100 connections Jul 12 18:04:34.929677 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: your mercy for graceful operations on workers is 90 seconds Jul 12 18:04:34.930025 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 18:04:34.930099 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: *** Operational MODE: preforking *** Jul 12 18:04:34.930360 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: *** uWSGI is running in multiple interpreter mode *** Jul 12 18:04:34.930360 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: spawned uWSGI master process (pid: 80565) Jul 12 18:04:34.930565 dv-cinder-844881-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 18:04:34.932411 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: spawned uWSGI worker 1 (pid: 80576, cores: 1) Jul 12 18:04:34.932411 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: spawned uWSGI worker 2 (pid: 80577, cores: 1) Jul 12 18:04:34.932411 dv-cinder-844881-1-iscsi devstack@keystone.service[80565]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 18:04:40.511375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 18:04:40.511375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: from cryptography.utils import int_from_bytes Jul 12 18:04:40.512165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 18:04:40.512165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: from cryptography.utils import int_from_bytes Jul 12 18:04:40.513920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 18:04:40.513920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: from cryptography.utils import int_from_bytes Jul 12 18:04:40.514368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 18:04:40.514368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: from cryptography.utils import int_from_bytes Jul 12 18:04:40.562344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 18:04:40.562559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 18:04:40.562750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] command line args: [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 18:04:40.562848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 18:04:40.563010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 18:04:40.563101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 18:04:40.563180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ================================================================================ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 18:04:40.563257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] command line args: [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 18:04:40.563428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] admin_token = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.563508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 18:04:40.563643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ================================================================================ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 18:04:40.563729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_dir = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.563845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.563928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] admin_token = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] config_source = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_dir = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] debug = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] config_source = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] debug = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.564922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.565963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_config_append = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_config_append = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_dir = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_dir = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.566980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.567940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.568943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.569936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] publish_errors = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.570921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] publish_errors = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.571908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] standard_threads = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] standard_threads = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.572884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_journal = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_json = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_stderr = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_journal = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] use_syslog = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_json = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.573923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.574037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_stderr = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.574149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.574262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] use_syslog = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.574381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.574506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 18:04:40.574621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.574734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.574848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.574966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.575918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.576892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.577951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.578998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.579994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.580976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.581090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.581697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.581824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.581938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.582979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.583943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.584886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.585934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.586953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.587987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.588945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.589889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.590950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.591963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.592960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.593893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.594959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.595914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.596936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.597588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.597714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.597831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.597946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.598905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.external = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.599939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.external = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.600917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.password = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] auth.token = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.password = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.601958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] auth.token = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.602906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.603972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.604953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.605837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.606970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.607946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.608953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.609903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.610955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.611886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.612001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.612117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.612232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.612368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.613901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.614965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.615914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.616920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.617979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.618902 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.619951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.620926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.621976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.622920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.623968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.624925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.625999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.626937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.627982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.628136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.628761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.628878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.629900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.630929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.631988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.632959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.633879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.634940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.635989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.636959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.637891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.638955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.639891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.640877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.641928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.642977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.643943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.644899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.645956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.646920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.647987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.648917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.649988 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80577) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 18:04:40.650561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.650561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.651900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 18:04:40.652076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80576) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 18:04:40.680452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.680674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.680893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.682461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.682700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.682933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.689194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.692080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.717552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.717731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.717955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.718193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.861606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.862182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.910908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.911133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.912295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.912560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.990402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.990555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.990770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.990991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.991792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.992016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:40.992237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.015848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.016988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.017070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.017202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.017292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.017472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.028336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.028873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.041537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.041771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.041995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80577) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.042947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.043112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.043332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.043551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80576) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 18:04:41.060626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.060833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 18:04:41.061082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.061298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 18:04:41.061396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.061580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 18:04:41.061853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.061957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 18:04:41.062147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.062838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.063096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.073453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.073724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.073989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.074072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.074268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.074355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.074581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.074864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.082676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.083294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.085972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.086245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.086522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.086604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.086798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.086896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.087944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.088768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.089000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.089270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.116287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.117549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.117906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.118184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.119184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.119470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.123427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.123701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.123976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.124800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.125077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.126089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.131398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.131676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.133112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.133398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.136135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.136447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.136726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.137000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.137268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.137532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.137795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.137921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.138056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.138209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.138493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.138778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.139058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.139334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.139605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.139871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.157491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.157491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.158661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.158940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.162429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.164083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.166042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.166334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.167835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.168133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.171451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.173335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.175799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.177595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.179459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.179753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.180039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.180396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.180646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.180923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.181268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.181572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.182158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.182449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.182743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.183023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.200224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.200527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.200792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.201052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.202308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.202596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.202868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.203132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.208993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.210707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.210978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.211181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.212935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.213217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.214281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.214561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.214955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.215242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.215520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.216474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.216758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.217044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.217326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.217604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.229569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.229845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.230121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.230404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.230686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.230965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.231233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.231940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.232224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.232532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.232816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.233099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.233387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.233666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.250886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.253914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.254152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.257274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.257470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.259061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.260627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.260863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.261140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.261428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.262244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.264029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.264326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.264619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.268775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.270486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.272252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.273726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.274013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.274097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.274291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.274569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.277303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.277599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.277888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.278166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.283750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.284039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.284865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.288174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.288488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.288781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.292908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80577) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.293186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.293463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.293734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.294003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.294277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.294549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.294820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.295186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.295452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.295727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.295989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.296252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.296541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80577) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.296710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80576) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 18:04:41.296991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.297275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.297553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.297830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.298120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.298398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.298679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.299063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.299348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.299628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.299895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.300786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.301217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80576) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 18:04:41.334960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560c62bcf9c0 pid: 80577 (default app) Jul 12 18:04:41.338332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e839d4-4944-4231-a85d-0bd63b5550d4 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:41.338630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e839d4-4944-4231-a85d-0bd63b5550d4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:41.338917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e839d4-4944-4231-a85d-0bd63b5550d4 None None] PATH_INFO: `/v3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:41.339761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560c62bcf9c0 pid: 80576 (default app) Jul 12 18:04:41.343166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Tue Jul 12 18:04:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:04:50.064618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:50.065193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:50.065193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:50.070282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:04:50.075941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:50.076460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:50.076591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:50.148953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: INFO dbcounter [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] Registered counter for database keystone Jul 12 18:04:50.171206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG oslo_db.sqlalchemy.engines [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80577) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 18:04:50.178273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writer thread running {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 18:04:50.358342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG passlib.handlers.bcrypt [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80577) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 18:04:50.358688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG passlib.handlers.bcrypt [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80577) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 18:04:50.414252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:50.415897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:04:50.541250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 18:04:50.749263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:50.822371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: INFO dbcounter [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] Registered counter for database keystone Jul 12 18:04:50.842307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG oslo_db.sqlalchemy.engines [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80576) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 18:04:50.850053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writer thread running {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 18:04:50.974170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327dea-ccdc-49cc-a01b-d7d9794437ea None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:50.978491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:50.979163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:50.979455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:50.979742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:50.989486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:51.042882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 18:04:51.043175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca3f8b-603e-4e97-b0dd-2641a235834a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:51.055154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Tue Jul 12 18:04:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:04:51.061300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:51.076132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4299e7-f233-4624-8be1-2ce595b70bcd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:51.080834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:51.082814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:51.083203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:51.083578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] PATH_INFO: `/v3/projects/cecdefe3d592442ba46417c3dd5f54f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:51.085172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] RBAC: Authorizing `identity:get_project(project_id=cecdefe3d592442ba46417c3dd5f54f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:51.137715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING keystone.common.rbac_enforcer.enforcer [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 18:04:51.138067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03454533-ef1f-4fff-b45c-f2920d8dfa98 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:51.142757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Tue Jul 12 18:04:51 2022] GET /identity/v3/projects/cecdefe3d592442ba46417c3dd5f54f9 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:04:53.237595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1da4ef-3201-4ff5-baab-3dac8a82342c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:53.238136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1da4ef-3201-4ff5-baab-3dac8a82342c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:53.238136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1da4ef-3201-4ff5-baab-3dac8a82342c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:53.242953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:04:53.248736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:53.249140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:53.249308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:53.268224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:53.269820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:04:53.278371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 18:04:53.302494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1da4ef-3201-4ff5-baab-3dac8a82342c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:53.314792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1da4ef-3201-4ff5-baab-3dac8a82342c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:53.319620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:53.322365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:53.322673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:53.322966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:53.333158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:53.334978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646cf89-dda1-4c39-b9cc-195b870e4cc4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:53.343971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Tue Jul 12 18:04:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:04:53.350832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:53.359997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05739bc0-b46a-4bbb-be62-b9e2ce9d8cf8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:53.364518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:53.365390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:53.365681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:53.365964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:53.367368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:53.369351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a4b720-872f-4a71-947f-896bb939d325 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:53.370926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Tue Jul 12 18:04:53 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:04:55.514396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172845-e9da-4305-80d4-cb4028538ba0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.514396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172845-e9da-4305-80d4-cb4028538ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.514396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09172845-e9da-4305-80d4-cb4028538ba0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.514396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:04:55.515402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.515402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.515402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.515402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:55.515402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c9fd-c234-41dd-9398-e0c01d62c900 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c9fd-c234-41dd-9398-e0c01d62c900 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7c9fd-c234-41dd-9398-e0c01d62c900 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.516254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.517045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.540879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG passlib.handlers.bcrypt [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80576) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 18:04:55.541301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG passlib.handlers.bcrypt [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80576) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 18:04:55.574614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:04:55.587449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:55.589207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:04:55.599052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:55.626862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:55.694181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:04:55.710226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02650f-0d3d-4734-86df-e08bad504b30 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:55.716806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:55.751565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:55.752379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.753366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.753660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.769851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:04:55.789852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:04:55.790856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:04:55.791771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:04:55.792432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:04:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:04:55.797755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:55.843689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bee49-de76-4b0d-a37f-286cca542dd4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:55.846153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:55.884380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:55.884617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:55.885521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.885821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.886106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.904230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96db9-bb91-46f5-bf76-b521c1f4f1a8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:55.939748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:55.940963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:55.941222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:55.941506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:55.942266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:04:55.942574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:55.944001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:55.955884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:04:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:04:55.958712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:55.960367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:55.961885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:56.087131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 18:04:56.088888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baaa9b4-f861-4b6f-8334-31b8274589dd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:56.092850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:56.095759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:56.124675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:56.125551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:56.125962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:56.126246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:56.127463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:56.129001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:56.130615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:04:55 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:04:56.135953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:56.151219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:56.199039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fbd8-7fc2-4bd2-8d88-35ec2de7370d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:56.235245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:56.235973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:56.236260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:56.236571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:56.242666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03050c0-ce26-4d8d-bc47-4721785d3833 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:56.255549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:56.257104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:56.279032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:04:56.279259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:04:56.280270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:04:56.281381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:04:56.281381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:04:56.281381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:04:56.281381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:04:56.281607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:56.281607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:56.281607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:56.281793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 9/18] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:04:56.281857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:56.283191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:56.285281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 10/19] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:04:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:04:56.286347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:56.291175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:56.348712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:56.378854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:04:56.395978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731f51-5e9d-4a15-ba0d-ac712730b2bb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:56.400246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ead44-ad69-487c-912e-f0aa53350f75 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:56.431443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:56.432118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:56.432436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:56.432722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:56.433453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:04:56.433764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:56.435160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e916ba12-c031-4cc2-a071-25e754f9c68a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:56.436436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:56.437289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:56.437638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:56.437990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:56.438916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.domain.id=default, user.id=26ab52b942da412ea520d26ff61916e1) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:04:56.439438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:56.441390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b867b-4fb2-4d88-8ab4-aeedce7efbf5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:56.452910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Tue Jul 12 18:04:56 2022] GET /identity/v3/role_assignments?user.id=26ab52b942da412ea520d26ff61916e1&scope.domain.id=default&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:04:56.455272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:04:59.323160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4224f-0669-4f50-9590-871615912a4f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:59.324692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4224f-0669-4f50-9590-871615912a4f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:59.325482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4224f-0669-4f50-9590-871615912a4f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:59.328295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:04:59.336068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551d42c-84ab-4c20-b219-8eb5f6e28f0e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:59.336834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551d42c-84ab-4c20-b219-8eb5f6e28f0e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:59.337174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551d42c-84ab-4c20-b219-8eb5f6e28f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:59.369839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b551d42c-84ab-4c20-b219-8eb5f6e28f0e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:59.371057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b551d42c-84ab-4c20-b219-8eb5f6e28f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:04:59.516059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 11/23] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:04:59.562569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4224f-0669-4f50-9590-871615912a4f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:04:59.826839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4224f-0669-4f50-9590-871615912a4f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:04:59.895913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b345-a3ba-4584-8df4-562aa4f4908d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:59.896440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b345-a3ba-4584-8df4-562aa4f4908d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:59.896942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3b345-a3ba-4584-8df4-562aa4f4908d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:59.899534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 12/24] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:04:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:04:59.906815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a5e1-07ba-4afd-aa21-5dce303707b6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:59.907316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a5e1-07ba-4afd-aa21-5dce303707b6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:59.907804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0a5e1-07ba-4afd-aa21-5dce303707b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:59.930260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:04:59.931336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:04:59.931847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:04:59.932369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:04:59.933819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:04:59.948534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:04:59.973956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-50c0a5e1-07ba-4afd-aa21-5dce303707b6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:04:59.978373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-50c0a5e1-07ba-4afd-aa21-5dce303707b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:00.085189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:00.173885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 13/25] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:00.176397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefde36-66f4-43fb-86b3-2119daf598c8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.176892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefde36-66f4-43fb-86b3-2119daf598c8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.177362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abefde36-66f4-43fb-86b3-2119daf598c8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.188850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 14/26] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:00 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:00.195837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.196342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.196828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.245859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:00.247707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:00.296345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 13/27] 10.222.0.47 () {60 vars in 1131 bytes} [Tue Jul 12 18:04:59 2022] POST /identity/v3/roles => generated 225 bytes in 735 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:05:00.298051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:00.313026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:00.461277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 15/28] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:00.535928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:00.620753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ea2d5-0c0e-4944-a05e-0ed5b365dfdf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:00.716921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:00.717927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.718590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.719067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.721121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:00.724601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:00.760737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a975ce1-6dc5-432c-8d5c-3945d1c448b4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:00.769355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:00.796021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:00.796941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.800917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.800917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.809270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-02e79b6f-44c2-4975-a28c-168795a06d57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:00.818336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:00.819984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 16/29] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:00.821934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd4e7-24e1-42c8-8494-c79906ca9353 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.823244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd4e7-24e1-42c8-8494-c79906ca9353 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.823244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcd4e7-24e1-42c8-8494-c79906ca9353 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.826416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 17/30] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:00.828550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e65c6-a36f-46f0-a112-109225e22e47 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.829196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e65c6-a36f-46f0-a112-109225e22e47 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.830003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e65c6-a36f-46f0-a112-109225e22e47 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.832446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 18/31] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:00.835008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a056e-51a1-4a03-8341-266a2ddb2f54 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:00.835349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a056e-51a1-4a03-8341-266a2ddb2f54 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:00.835534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a056e-51a1-4a03-8341-266a2ddb2f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:00.837323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 14/32] 10.222.0.47 () {60 vars in 1131 bytes} [Tue Jul 12 18:05:00 2022] POST /identity/v3/roles => generated 219 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:00.838411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:00.884146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-029a056e-51a1-4a03-8341-266a2ddb2f54 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:00.886777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-029a056e-51a1-4a03-8341-266a2ddb2f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:00.935208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:00.986064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bef6e-b28b-4a55-96b1-1c6c34789843 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.039501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 19/33] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:01.039899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.041108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.041634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.041888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.042143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.042572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.043068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.043571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:01.044121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.046574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.068203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 15/34] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:01.068614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:01.070060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:01.075961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.168067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 20/35] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:01.169294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.218985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:01.223015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc053-97e7-4d33-9049-43f4012e4684 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.251355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:01.260704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.264464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.264464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.264464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.264464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.264464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.298182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26dc1-0646-440d-9c98-38a7dfb3e1a7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.332693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.333578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.333963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.334354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.336718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.338217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184b0d3-7a39-454b-8d6f-1eae1343294c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.339851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 21/36] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:01 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:01.341210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eae694-87e9-4aa6-9064-3b93fc8ac972 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.341507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eae694-87e9-4aa6-9064-3b93fc8ac972 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.341792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eae694-87e9-4aa6-9064-3b93fc8ac972 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.343465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 22/37] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:01.344984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eae694-87e9-4aa6-9064-3b93fc8ac972 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.352233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 16/38] 10.222.0.47 () {60 vars in 1131 bytes} [Tue Jul 12 18:05:01 2022] POST /identity/v3/roles => generated 223 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:01.353097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.470897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:01.481707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eae694-87e9-4aa6-9064-3b93fc8ac972 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.507076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da03ca1d-da63-497d-9655-2b3fb8bd81fe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.540541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.541615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.542015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.542498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.549570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.550860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.551257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.551599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.553174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.555663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f1b3-c02b-49b1-828e-54cbd2afded9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.558021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 17/39] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:01.558334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.560141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.560728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.561123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.561530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:05:01.584574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:01.586126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:01.587715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:05:01.587715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:05:01.587715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:05:01.587715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:01.587715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 23/40] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:01.588197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:01.588361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.589599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:01.658431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:01.677252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 18/41] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:01.678125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.729293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368aaab-7623-4aa8-81de-330a9b5ffa3f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.775118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.775920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.776210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.776525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] PATH_INFO: `/v3/domains/default/users/26ab52b942da412ea520d26ff61916e1/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.779219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=26ab52b942da412ea520d26ff61916e1, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.780745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:01.807680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd24344-ed66-46ae-af3f-9dd546235738 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:01.984934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 24/42] 10.222.0.47 () {58 vars in 1361 bytes} [Tue Jul 12 18:05:01 2022] PUT /identity/v3/domains/default/users/26ab52b942da412ea520d26ff61916e1/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:01.986191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:01.992159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:01.993221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:01.993663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:01.994127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:01.995359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:01.995838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:01.997050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:01.998246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:02.033627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 19/43] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:02.036517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:02.107263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376b7ba-55ce-41b8-a24f-cbcbfcd03505 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:02.111380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:02.162736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:02.163554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:02.163849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:02.164142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:02.165509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:02.167172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:02.171779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 25/44] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:02.177562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:02.184431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf41a68-39dd-436d-ac7b-a9e6600e519b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:02.225008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:02.226045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:02.226495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:02.226925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:02.228169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:02.230599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04285e03-d977-4fba-808d-8db5d95ce9ae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:02.265904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:02.292031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d766de4-6eb4-4427-bf38-144dc358572b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:02.365308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 20/45] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:05:02 2022] POST /identity/v3/projects => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:05:02.439371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:02.440970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:02.440970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:02.441230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:02.444771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:02.444771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6270cec-e45c-4b2f-b189-8d25a1bf3407 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:02.539813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 26/46] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:05:02 2022] POST /identity/v3/projects => generated 312 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:05:04.044632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc407-b5e9-45b3-86fe-34c7753e0e2f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.045148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc407-b5e9-45b3-86fe-34c7753e0e2f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.045258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc407-b5e9-45b3-86fe-34c7753e0e2f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.046591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 21/47] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:04.052524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.052955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.053116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.070627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:04.072204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:04.139628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 27/48] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:04.162634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cc407-b5e9-45b3-86fe-34c7753e0e2f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.258265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.264851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cc407-b5e9-45b3-86fe-34c7753e0e2f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.299772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.300573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.300867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.301150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.311189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:04.320276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:04.321163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:04.321933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 22/49] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:04.327284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.383098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.426964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d61453-64f9-48da-92b5-2338bbe70ab0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.463048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.463731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.464020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.464318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.465092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:04.465406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.466814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb314ce-409c-4d76-9789-504c64e53455 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:04.479749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 28/50] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:04.485289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.563027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a10849-b8e3-454d-bdbe-51323218ae89 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.563256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a10849-b8e3-454d-bdbe-51323218ae89 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.563546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a10849-b8e3-454d-bdbe-51323218ae89 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.565047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 29/51] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:04.570103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.570404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.570693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.586606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db02a59-a85b-4878-af23-7b0216309e83 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.587905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:04.589183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:04.623365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.624252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.624576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.624862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.626091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.627638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:04.629496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 23/52] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:04.634831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.660352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 30/53] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:04.664654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.682962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.734744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.741312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc2995-e2e3-40c4-8756-8197a8d51c5c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.778661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.779476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.779761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.780038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.781316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.782906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:04.785252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 24/54] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:04.788116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c47064-bf3c-4d33-bca5-bc5c036aa8e6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.791292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.824795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.825524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.825819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.826105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.827328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.828884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:04.831275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 31/55] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:04.836976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:04.886205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.895693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931e05e-4657-4488-8fc5-c6ec3542d141 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.931122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.931775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.932069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.932382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.933251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.domain.id=default, user.id=26ab52b942da412ea520d26ff61916e1) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:04.933570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.935045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45d23-0e91-45ca-ac28-a44bb33a1dd8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:04.945279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132d56e-21ad-4c66-9e4e-4bc8cd2d977e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:04.946278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 25/56] 10.222.0.47 () {56 vars in 1322 bytes} [Tue Jul 12 18:05:04 2022] GET /identity/v3/role_assignments?user.id=26ab52b942da412ea520d26ff61916e1&scope.domain.id=default&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:04.967120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:04.988232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:04.988968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:04.989260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:04.989571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:04.990320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:04.991814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87226494-88b9-49af-a304-182cd6102ecd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:05.084359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 32/57] 10.222.0.47 () {60 vars in 1140 bytes} [Tue Jul 12 18:05:04 2022] POST /identity/v3/projects => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:05:07.117647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef1f46-72c3-41b2-b103-bfb5c9d861e6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:07.118223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef1f46-72c3-41b2-b103-bfb5c9d861e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:07.118223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef1f46-72c3-41b2-b103-bfb5c9d861e6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:07.119584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 26/58] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:07.124653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:07.125028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:07.125229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:07.142283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:07.143704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:07.207010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 33/59] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:07.231565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef1f46-72c3-41b2-b103-bfb5c9d861e6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:07.329000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef1f46-72c3-41b2-b103-bfb5c9d861e6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:07.361952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:07.362729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:07.363011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:07.363291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:07.364496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:07.365960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1896a660-b0ef-4253-83ce-362a9c6b8a09 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:07.367559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 27/60] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:07.373097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:07.468657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:07.472837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e2927-dacf-418c-bd6d-18ef5a9f2c7b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:07.506601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:07.507280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:07.507577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:07.507871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:07.508655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:07.510150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c5a6e-1988-4e15-99d1-d8f11ce2305b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:07.618329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 34/61] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:07 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:10.884531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391dc8d-a214-4dec-80a9-3c1966748810 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:10.885030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391dc8d-a214-4dec-80a9-3c1966748810 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:10.885030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391dc8d-a214-4dec-80a9-3c1966748810 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:10.886615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 28/62] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:10.898020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5f1bd-8871-4aaa-a59a-32c46209aca7 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:10.898174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:10.898570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:10.898639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:10.899191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5f1bd-8871-4aaa-a59a-32c46209aca7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:10.899701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff5f1bd-8871-4aaa-a59a-32c46209aca7 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:10.901504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 29/63] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:10.907203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:10.907767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:10.908256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:10.917001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:10.918489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:10.939282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:10.941444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:10.963273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.029096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 35/64] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:11.058626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 30/65] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:11.071911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.086951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.203928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff123ef-169b-41dd-8808-765c827eb7f2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.215657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9731bb9-5fdb-461c-8e96-82a94d2d3ba0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.217670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.242851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.243977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.244483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.244958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.253159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.254255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.254697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.255125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.256732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3988def7-b247-4c4b-89bb-6a5f602ed2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:11.265667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:11.266662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:11.267575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:11.268892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:11.270306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 36/66] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:11.271882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b52e2-2310-4eaa-aa6c-41e0bfefdfdc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.272331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b52e2-2310-4eaa-aa6c-41e0bfefdfdc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.272754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b52e2-2310-4eaa-aa6c-41e0bfefdfdc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.283565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 37/67] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:11.283565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b52e2-2310-4eaa-aa6c-41e0bfefdfdc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-afb57d91-31ad-4a89-b0e4-56e2d2614ce4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:11.283735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:11.284621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:11.285496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 31/68] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:11.288919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.289228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.289522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.307681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:11.309024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:11.320163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.384335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 32/69] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:11.385302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.396844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b52e2-2310-4eaa-aa6c-41e0bfefdfdc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.435155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.435849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.436137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.436457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.437233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:11.437552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:11.439094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05923ad-837c-4b1d-9a82-7f3e5eb83f02 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:11.457542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 38/70] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:11.463301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36145-9b41-4393-aaf0-e8219646867d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.463766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36145-9b41-4393-aaf0-e8219646867d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.464057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f36145-9b41-4393-aaf0-e8219646867d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.465385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 39/71] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:11.468464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36145-9b41-4393-aaf0-e8219646867d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.476494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.513964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cc35e-d0e2-4365-982b-4e2d1da047ae None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.547491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.548169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.548491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.548776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.550264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:11.550578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:11.551990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:11.559879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 33/72] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:11.560691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.569933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.570759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f36145-9b41-4393-aaf0-e8219646867d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.605301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.606217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.606508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.606794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.617507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1e8bd75d-e83b-4299-939b-98e3837738b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:11.625640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:11.626613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:11.628113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 40/73] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:11.628865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.629042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.629286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.645603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:11.646803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:11.668341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a79799-d44a-4583-96a4-cad7d7da8278 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.702219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.703099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.703389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.703667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.704913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:11.706413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.706520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb7fb9-6cc1-45fd-9b22-4a4ea10d7551 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:11.708126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 34/74] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:11.709278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ed27-1c40-46f7-a263-c98efdd1d4e2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.709577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ed27-1c40-46f7-a263-c98efdd1d4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.709859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ed27-1c40-46f7-a263-c98efdd1d4e2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.711252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 35/75] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:11.711777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029ed27-1c40-46f7-a263-c98efdd1d4e2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.717309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 41/76] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:11.727240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.797642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.818213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aa684-6403-4049-be6a-13e733c8205b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.823129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029ed27-1c40-46f7-a263-c98efdd1d4e2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:11.852560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.853245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.853542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.853838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.854610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:11.854932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:11.856366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:11.857527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:11.858405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.858699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.858984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.864963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 42/77] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:11.865778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.873470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:11.874956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7357e-21e2-4e0f-96bf-5c43f942e982 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:11.876910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 36/78] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:11.878723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:11.879019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:11.879305 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:11.895992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:11.897249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:11.927519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:11.964428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 37/79] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:11.964428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:11.968791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071109d-56fd-4401-8d35-0db3da8593a1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.004832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.005489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.005778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.006058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.025144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.065010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1422444-c545-4051-93f1-f1dccc0b7475 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.071804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.073330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.099020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.099818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.100109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.100424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.110577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:12.118468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:12.119468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:12.120409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 38/80] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:12.121102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.132936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.175942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 43/81] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:12.176830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.221944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b4205-cbb5-4246-8a6d-024c49c6702e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.244767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.256939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.257810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.258096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.258374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.259582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.261178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.262798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 39/82] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:12.263563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.279918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f818a-8b8e-4da3-ab7d-2b28372bd0dd None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.315768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.316462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.316756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.317044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.343537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.362061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c9a8a-329f-40f3-9f4b-6310d4fec991 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.386918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.388507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.396042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.396907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.397216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.397500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.398703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.400158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.401825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 40/83] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:12.402653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.461898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.496207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 44/84] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:12.497024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.505019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06806632-f02f-4c5a-a9b4-e40a74febe0a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.540762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.541430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.541721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.542012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.542771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:12.543087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.544501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.553265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 41/85] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:12.554053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.554986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.599423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d198820-2a45-49af-9056-2dbc69978868 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.633183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.634049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.634348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.634635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.635836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.637333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.639039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 45/86] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:12.639798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.653461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3f834-cd25-4997-aa87-16bb0255c4d7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.672835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.686957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.687611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.687901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.688185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.740440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da458028-6150-414e-b569-1fbee9a66da6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:12.754201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.755625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.766040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.774333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:12.774965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:12.775256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:12.775537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:12.776265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:12.777742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:12.854177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 42/87] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:12.855137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.875805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:12.878820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 46/88] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:12 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:12.879694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:12.968745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f58c61-08d3-4c90-ba1f-2967edb5079a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.002992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4cc64-d9df-4532-8c59-186866fb3a26 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.008871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.009965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.010417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.010841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.012337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.014003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.015929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 43/89] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.016932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.048958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.050032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.050228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.050559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.051818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.053336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.059293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 47/90] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:12 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.059293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.122690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:13.129539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:13.141352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf3670-3916-4622-b464-6faf0aee873f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.188946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3613b30-fdb6-4360-af9c-4dc9c2032b8f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.190086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.191084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.191449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.191808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.194168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=26ab52b942da412ea520d26ff61916e1) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:13.194281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.195825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.202046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 44/91] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/role_assignments?user.id=26ab52b942da412ea520d26ff61916e1&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.202905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.236381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.237365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.237663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.237956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.239629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.241234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.243289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 48/92] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.246739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.347528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5a59a-0a97-4ad2-8a5b-bcf30aca820a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.405012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.407329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.407933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.410339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.411665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:13.412892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.415225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.417139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bde69-e4b5-4e0e-ad57-a92dbaf70a0b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.432103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 45/93] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=5a2774adec8f4206a81f758406367b9f => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.433254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.450063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:13.459233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:13.470505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.471754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.472043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.472406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.559230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d6f8f-2211-4626-a4e1-1e938de21982 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.605942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.606517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.606628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.606882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.607175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.608039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.608146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:13.608393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.610503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.618963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 46/94] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.733249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 49/95] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:13.739048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:13.888848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:13.915916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1b7d3-cc76-49f5-a4b5-18180ed17dcf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:13.965337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:13.966466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:13.966812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:13.967151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] PATH_INFO: `/v3/projects/119df37b12e44827b73bf2cf2fe8a819` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:13.973490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] RBAC: Authorizing `identity:get_project(project_id=119df37b12e44827b73bf2cf2fe8a819)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:13.975260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc4e5-9605-4289-8710-2d0308330224 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:13.977405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 47/96] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/projects/119df37b12e44827b73bf2cf2fe8a819 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:13.983588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:14.021837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:14.123469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52225-ab75-4cc8-bfbd-c255f2ce6d77 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:14.163921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:14.164541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:14.164835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:14.165126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:14.165985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=119df37b12e44827b73bf2cf2fe8a819, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:14.166301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:14.167785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b850af3-47d7-49c8-ab75-1364f345ba63 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:14.176920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 50/97] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:13 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=119df37b12e44827b73bf2cf2fe8a819&role.id=5a2774adec8f4206a81f758406367b9f => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:15.855103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978939d2-69c0-46c1-b482-b23c3871f296 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:15.855950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978939d2-69c0-46c1-b482-b23c3871f296 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:15.856493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978939d2-69c0-46c1-b482-b23c3871f296 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:15.858016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 48/98] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:15.864130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163fd08-da91-4411-bc99-9fddd692531b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:15.864915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163fd08-da91-4411-bc99-9fddd692531b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:15.865348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163fd08-da91-4411-bc99-9fddd692531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:15.967115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2163fd08-da91-4411-bc99-9fddd692531b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:15.968587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2163fd08-da91-4411-bc99-9fddd692531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:16.058266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 51/99] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:16.109922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978939d2-69c0-46c1-b482-b23c3871f296 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:16.134880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fac8a-3f81-4633-8371-d8ab4d9dc578 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.135422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fac8a-3f81-4633-8371-d8ab4d9dc578 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.135853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fac8a-3f81-4633-8371-d8ab4d9dc578 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.137416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 52/100] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:16.142263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.143237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.143670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.166555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:16.168715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:16.187058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:16.276257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 53/101] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:16.316065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:16.319552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978939d2-69c0-46c1-b482-b23c3871f296 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:16.433084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:16.433917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.434212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.434499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.435918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:16.438395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424498e-f1b0-4740-a639-061bc168f8f5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:16.441231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 49/102] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:16.442676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5e6c8-66ae-4f8f-b7b7-301b474e3262 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.443118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5e6c8-66ae-4f8f-b7b7-301b474e3262 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.443629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5e6c8-66ae-4f8f-b7b7-301b474e3262 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.445327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 50/103] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:16.452396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5e6c8-66ae-4f8f-b7b7-301b474e3262 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:16.481073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:16.486442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c11a2-9951-4421-b058-720694f14f0d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:16.526252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:16.527448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.527912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.528392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.540875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-900c3deb-fce6-475d-a666-95d220def95e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:16.550402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:16.551263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:16.552035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:16.554648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 54/104] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:16.556386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18321012-51a9-457d-867f-1b21771c9665 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.556879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18321012-51a9-457d-867f-1b21771c9665 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.557347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18321012-51a9-457d-867f-1b21771c9665 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.559056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 55/105] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:16.563416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.563416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.563416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.568711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:16.583355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:16.583731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5e6c8-66ae-4f8f-b7b7-301b474e3262 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:16.585287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:16.638060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:16.638951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.639254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.639589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.640740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:16.642567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f4177-15cd-4ee6-8c9d-dc0ce291dd4c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:16.668411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 56/106] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:16.669430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:16.744414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 51/107] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:05:16 2022] POST /identity/v3/projects => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:16.744414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.744414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.744414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.780992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:16.784402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:16.839942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702ab9b-fefe-46db-af7f-8c0d15f9517a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:16.907820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:16.918387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:16.920615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.921336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.922471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.923442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:16.923762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:16.925939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d2cb-f751-4fb8-9f9e-e1c36d1cfff5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:16.927876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:16.934354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 52/108] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:16.935518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:16.937440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 57/109] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:16.939173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc6b2-d2e6-4449-bdf4-17ac8bbf6a68 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.943994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc6b2-d2e6-4449-bdf4-17ac8bbf6a68 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.943994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdc6b2-d2e6-4449-bdf4-17ac8bbf6a68 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.943994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 58/110] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:16.946640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:16.947266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:16.947558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:16.967285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:16.968436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:17.048799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671f9bb-c889-4d54-b8d7-8abe08448b46 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.053948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 59/111] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:17.054811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.087415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.088429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.088723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.089001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.100716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9dc0e9b3-87b2-4010-9568-61a3e5af17a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:17.109927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:17.110838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:17.112752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 53/112] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:17.117066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655e7d-25a1-419a-8ffa-28d80c0b3bdc None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.117066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655e7d-25a1-419a-8ffa-28d80c0b3bdc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.117066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78655e7d-25a1-419a-8ffa-28d80c0b3bdc None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.117066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 54/113] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:17.117779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78655e7d-25a1-419a-8ffa-28d80c0b3bdc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.174239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f8537-ba2e-4312-87b4-541cf6c0c819 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.193521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.199433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.211276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.212166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.212492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.212783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] PATH_INFO: `/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.228847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] RBAC: Authorizing `identity:get_user(user_id=9945ba10ce7b4d8797cc313f1dc3f7a7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.230397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:17.232228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 60/114] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:17.232228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.243903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78655e7d-25a1-419a-8ffa-28d80c0b3bdc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.284163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.285122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.285413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.285698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.297043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:17.306140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:17.306995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 55/115] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:17.312972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.346559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323cfe7-7c87-46ee-9ed1-bbe8c265a815 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.381728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.382538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.382830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.383142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.393820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-aaa5c904-2c29-45fb-a206-4cf3df1f15b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:17.401995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:17.402851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:17.403610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 61/116] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:17.405163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.405163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.405163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.421858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:17.423065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:17.429904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4522d5-5686-453a-9d64-d52b7658c66a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.456497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.464430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.475438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.476122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.476454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.476842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.477519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:17.477836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.479253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:17.492344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 56/117] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:17.492344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.511286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 62/118] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:17.511286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.609518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d12cd7-490a-4a1b-b032-2f77c272b372 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.622224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd403749-3af1-4e19-b62c-f7c431e32152 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.645873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.646727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.647016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.647467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.660108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.662378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.662378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.662378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.662378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:17.662838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.663983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:17.672713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 63/119] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:17.673510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.704568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.710256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.726365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.727911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:17.779241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2eda1-ed47-4ace-8a02-f0bf82b04a2e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.818888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.819835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.820126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.820450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.830857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:17.830857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.830857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:17.832353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 64/120] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:17.837288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.842555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 57/121] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:17.847019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:17.932863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.951067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:17.952862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af611bb-1f77-4abe-b03a-e4e757807bd1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.958881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1244e-5f1c-40b1-a34d-1f3414e963f0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:17.989206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.990026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.990315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.990602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.994649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:17.995528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:17.995818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:17.996100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:17.997372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:17.998872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.004907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 65/122] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.004907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.006961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:18.014885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:18.015782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:18.016618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 58/123] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:18.017295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.118759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f66d5-cfe4-49da-a47f-1a3544925a6a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.129514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780668f0-3d9a-4307-9801-615da1c80676 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.154515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.155402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.155701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.155989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.157236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.158903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.164338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 66/124] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.164338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.172712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.173595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.174391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.174677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.175937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.177453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.179087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 59/125] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.179847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.192869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.201519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.275172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f789ef-7d11-4d7e-81b1-2f6ce15fa151 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.284575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a047f-5a53-421f-86d6-c6eb6f13dd0d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.312319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.313409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.313716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.314000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.315861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.317375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.319462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.320364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.320664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.320953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.322169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.323642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.324085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 67/126] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.325000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.325864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 60/127] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.326450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.406343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.428750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.432046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06731a-d028-47b2-b6c5-bad4e3c6eaaf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.432251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af1578-0a12-4c39-82e3-5aceb685a06a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.474480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.475364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.475664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.475957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.476061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.476816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.477113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.477214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.477398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.478160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:18.478473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.478654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.479864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.480565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 61/128] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.481203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.493785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 68/129] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.494632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.591462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673cb62a-9576-4d28-8bd3-8047f9e66062 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.602297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c54fb3-ca1e-4ec7-8536-b964849634be None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.628776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.629650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.629941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.630234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.631439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.632949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.634655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 62/130] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.635396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.643282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.643954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.644246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.644582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.645448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=9945ba10ce7b4d8797cc313f1dc3f7a7) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:18.645769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.647242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.650978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.653058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 69/131] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/role_assignments?user.id=9945ba10ce7b4d8797cc313f1dc3f7a7&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:18.665559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.715846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:18.757075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d587fec-8d59-4fd3-854b-7bb3bc80e7e6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.797186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.798563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.799091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.799584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/2a598a92f6ea439a98503435cf969703/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.802061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, user_id=2a598a92f6ea439a98503435cf969703, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.803814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.821733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a6b80-8f1f-43bd-b513-bcaea8aa4996 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:18.857751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:18.858659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:18.858955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:18.859239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/26ab52b942da412ea520d26ff61916e1/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:18.861827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, user_id=26ab52b942da412ea520d26ff61916e1, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:18.865855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:18.896377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 63/132] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:18 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/2a598a92f6ea439a98503435cf969703/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:18.896377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.961970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 70/133] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:18 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/26ab52b942da412ea520d26ff61916e1/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:18.962844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:18.981657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:19.034852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42fa5-8295-4518-a6db-96961c2cd3cd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.077872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:19.095393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7e544-b4e9-4460-a415-cd6c2f85a9a6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.116077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.117290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.117583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.118062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.119701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.121292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623c786-4cba-43ea-99f0-f2b5ebbebd6c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:19.124132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 64/134] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:18 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:19.125612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a91-03e2-4622-aa3a-baab5df3c975 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.129778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a91-03e2-4622-aa3a-baab5df3c975 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.129778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba7a91-03e2-4622-aa3a-baab5df3c975 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.129778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 65/135] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:19.130104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7a91-03e2-4622-aa3a-baab5df3c975 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:19.141480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.142392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.143275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.143570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/2a598a92f6ea439a98503435cf969703/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.146005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, user_id=2a598a92f6ea439a98503435cf969703, role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.147502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422e1c2-9180-400a-8b09-6d899fa9f469 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:19.224536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 71/136] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:18 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/2a598a92f6ea439a98503435cf969703/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:19.226847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.227150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.227440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.265113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:19.266810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:19.301259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba7a91-03e2-4622-aa3a-baab5df3c975 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.384129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.387911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.388537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.389021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] PATH_INFO: `/v3/projects/119df37b12e44827b73bf2cf2fe8a819` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.390650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] RBAC: Authorizing `identity:get_project(project_id=119df37b12e44827b73bf2cf2fe8a819)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.394208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:19.400519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 66/137] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:19 2022] GET /identity/v3/projects/119df37b12e44827b73bf2cf2fe8a819 => generated 312 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:19.407687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:19.423319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 72/138] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:19.430169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:19.525420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b58a5-d680-492a-82d1-98fc6873235e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.526729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:19.609609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.610916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.611956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.613113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] PATH_INFO: `/v3/projects/119df37b12e44827b73bf2cf2fe8a819/users/2a598a92f6ea439a98503435cf969703/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.618812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] RBAC: Authorizing `identity:create_grant(project_id=119df37b12e44827b73bf2cf2fe8a819, user_id=2a598a92f6ea439a98503435cf969703, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.620662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d6a0b-07c4-4383-b555-5e872091f577 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:19.649854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:19.674638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287e969-ecfd-4c9e-b77f-782410d305c7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.711692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.712580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.712873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.713154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.714381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.715836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:19.717629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 73/139] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:19.724598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:19.745888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 67/140] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:19 2022] PUT /identity/v3/projects/119df37b12e44827b73bf2cf2fe8a819/users/2a598a92f6ea439a98503435cf969703/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:19.857747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf16545-e9fa-4314-9d1e-e11ea1036532 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:19.911193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:19.911885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:19.912173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:19.912530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:19.913316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:19.914775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e51ddc-2ae6-4f42-849c-f52a1811aa15 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:20.011661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:20.016459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 74/141] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:19 2022] POST /identity/v3/users => generated 283 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:21.452758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448ee0-d6f2-4385-806a-cf48b7b2675e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:21.453691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448ee0-d6f2-4385-806a-cf48b7b2675e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:21.454177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69448ee0-d6f2-4385-806a-cf48b7b2675e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:21.456170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 68/142] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:21.461684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40aef-73ba-4f78-ab93-634a6553723a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:21.462124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40aef-73ba-4f78-ab93-634a6553723a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:21.462293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c40aef-73ba-4f78-ab93-634a6553723a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:21.484584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-34c40aef-73ba-4f78-ab93-634a6553723a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:21.485795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-34c40aef-73ba-4f78-ab93-634a6553723a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:21.577618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 75/143] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:21.614814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69448ee0-d6f2-4385-806a-cf48b7b2675e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:21.718150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:21.761578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bce0d3-991a-41bc-917d-0cfc0f3ee9ec None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:21.762030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bce0d3-991a-41bc-917d-0cfc0f3ee9ec None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:21.762414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bce0d3-991a-41bc-917d-0cfc0f3ee9ec None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:21.764047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 76/144] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:21.769975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69448ee0-d6f2-4385-806a-cf48b7b2675e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:21.775506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:21.775942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:21.780859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:21.803274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:21.806335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:21.818463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:21.819519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:21.819994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:21.820454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:21.849878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:21.857723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:21.858797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:21.859750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 69/145] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:21.866620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:21.955492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 77/146] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:21.995329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.011733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1c2b4-41b9-4fcb-b1d9-60d829d594d3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.049036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:22.058148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.058834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.061335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.061335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.061335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:22.061335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:22.062920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:22.074745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 70/147] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:22.081055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.130280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:22.211356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23609b-1bbb-4a4e-b83f-6edb49803575 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.211717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f4665-8c27-4747-956c-d7b2211db34d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.251627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.253617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.253617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.253617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.277532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b38c24c0-11d1-42fe-a5fd-87821d4ea106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:22.287340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:22.288205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:22.289029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:22.291512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 78/148] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:22.297196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141e3b-5522-404c-b158-a5ad0dacd484 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.297749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141e3b-5522-404c-b158-a5ad0dacd484 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.298252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e141e3b-5522-404c-b158-a5ad0dacd484 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.301600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 79/149] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:22.302612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e141e3b-5522-404c-b158-a5ad0dacd484 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.310737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.310737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.310737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.310737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] PATH_INFO: `/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.313080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] RBAC: Authorizing `identity:get_user(user_id=9945ba10ce7b4d8797cc313f1dc3f7a7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:22.315630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba1b27-f0fa-4302-8b45-ae3329f1053d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:22.316884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 71/150] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:22 2022] GET /identity/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7 => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:22.318084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.318376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.318660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.347316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:22.348861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:22.453396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:22.477719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 72/151] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:22.478589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.514207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e141e3b-5522-404c-b158-a5ad0dacd484 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.551684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:22.578875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.579567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.579932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.581833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.582553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:22.582875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:22.585082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ffd63-5451-4a8f-8a2e-a30840a77d92 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:22.598614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 80/152] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:22.604675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef39a-c458-4758-b7b9-f0d378eac1c8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.604931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef39a-c458-4758-b7b9-f0d378eac1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.605209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ef39a-c458-4758-b7b9-f0d378eac1c8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.608262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 81/153] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:22.609066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ef39a-c458-4758-b7b9-f0d378eac1c8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.671707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a6926-006a-41bb-bab6-994b357b0742 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.739379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.740415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.740888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.741178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.743493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:22.745408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:22.747682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 73/154] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:22 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:22.748502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.849644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:22.872770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ef39a-c458-4758-b7b9-f0d378eac1c8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.925617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e1795-41cb-42d9-9958-2e4b96064ec7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:22.951895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.953276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.953784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.954275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.981253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:22.988817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:22.989768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.994798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.994798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] PATH_INFO: `/v3/users/26ab52b942da412ea520d26ff61916e1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:22.994798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=26ab52b942da412ea520d26ff61916e1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:22.994798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b6400-18ec-4918-9d69-462ce945bcb1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:22.995108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:22.996513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:22.997335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:22.997927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 74/155] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:22 2022] GET /identity/v3/users/26ab52b942da412ea520d26ff61916e1 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:22.997993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 82/156] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:22.998056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:22.998118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:22.998180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:22.998180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.025155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:23.026518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:23.088783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:23.163116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 75/157] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:23.163987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:23.201252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223067b2-f8b8-4740-aa5f-bddbbdcc32ad None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:23.243917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:23.290043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed7b94-ccab-409a-a07d-aac89e92fdfc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:23.291925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:23.293246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.293741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.294228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/9945ba10ce7b4d8797cc313f1dc3f7a7/roles/d54be0042c4c4a2fb1380e52eb16b570` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.296500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, user_id=9945ba10ce7b4d8797cc313f1dc3f7a7, role_id=d54be0042c4c4a2fb1380e52eb16b570)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:23.298212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ed18e-405f-44a2-99ec-abedfc6eb14d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:23.324992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:23.333092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:23.337416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 76/158] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:23.338354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:23.405153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 83/159] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:22 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/users/9945ba10ce7b4d8797cc313f1dc3f7a7/roles/d54be0042c4c4a2fb1380e52eb16b570 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:23.412350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a930be7-8482-481a-a409-a18d961a87a8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.413366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a930be7-8482-481a-a409-a18d961a87a8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.413936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a930be7-8482-481a-a409-a18d961a87a8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.418608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 84/160] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:23.419850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a930be7-8482-481a-a409-a18d961a87a8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:23.452430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545de6d-5b59-4931-992c-9244e5e459b1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:23.469514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:23.495655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:23.496406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.496695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.496971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.586911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a930be7-8482-481a-a409-a18d961a87a8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:23.588690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:23.592850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:23.594447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9aa1a7-0d65-4ade-afe6-9b2d401ee912 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:23.657239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:23.658699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.659166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.659449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.690219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:23.704337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:23.705326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:23.706137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:23.708933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 85/161] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:23.710003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:23.771105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 77/162] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:23.773299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.773603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.773888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.801887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:23.804710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:23.831232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:23.908849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c03ff7-5d32-4b4f-9294-efa60c0be427 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:23.927859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 78/163] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:23.928755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:23.989134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:23.990042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:23.990335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:23.990618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:23.991909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:23.993420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:23.995140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 86/164] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:23.995965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.031733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f371738-16c3-4273-b869-2a430a26a892 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.062456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.068845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.069525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.069811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.070087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.070846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:24.071154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.072568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.081360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 79/165] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:24.082098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.097726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.195070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff51004-50f2-4c66-89bb-a816ace0ea3e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.231414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.232364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.232668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.232949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.245036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:24.250721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a603-a69d-4998-83b1-2861e498f1fe None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:24.253750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:24.254624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:24.255421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 80/166] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:24.256079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.359857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.360775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.361075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.362403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.370009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.373035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.374841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 87/167] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:23 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:24.375626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.401036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.461521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392f8b-d1f8-4aea-b429-f93554417a9e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.530629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.531252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.531496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.531782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.577964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.590463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e5f8e-8a8a-4c0f-8881-debd9ac056a3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.620534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.622264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.700974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.701610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.703512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.706270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.707129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.713033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.714783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.718864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 88/168] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:24.719901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.765989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 81/169] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:24.769608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.863782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf487-c119-4af6-be1c-390774b13742 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.902713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.903389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.903674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.903954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.904755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:24.905061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.906444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47ae49-8c6d-4fd6-b636-775bae39eb23 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.906842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d63a1b-ef6d-49cd-8e19-8aaf79a6b5e5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:24.915033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 89/170] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:24.917624 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33263261-4b9d-4fcd-9095-409fc2484a6f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.917919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33263261-4b9d-4fcd-9095-409fc2484a6f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.918199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33263261-4b9d-4fcd-9095-409fc2484a6f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.919478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 90/171] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:24.920183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33263261-4b9d-4fcd-9095-409fc2484a6f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:24.951562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:24.952677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:24.953341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:24.953627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:24.953904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:24.954746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=26ab52b942da412ea520d26ff61916e1) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:24.955094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:24.956769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:24.963397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 82/172] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/role_assignments?user.id=26ab52b942da412ea520d26ff61916e1&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:24.964182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.019170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.044580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33263261-4b9d-4fcd-9095-409fc2484a6f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.094045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.095508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.096117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.096580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.114884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40cec30-15ee-4ac4-85ef-0ccbf56500b4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.165763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.166674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.166950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.167231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.168608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.170126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78ef8-7e36-4ec1-af77-bc9dac8365ef None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.172284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 83/173] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:25.174058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23855-a073-4189-8fd5-f8fc9cd494bf None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.174350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23855-a073-4189-8fd5-f8fc9cd494bf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.174662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c23855-a073-4189-8fd5-f8fc9cd494bf None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.176185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 84/174] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:25.178349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa07b-9a56-47ec-9aed-b426c9cb96e9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.178764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa07b-9a56-47ec-9aed-b426c9cb96e9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.178933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa07b-9a56-47ec-9aed-b426c9cb96e9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.180330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 85/175] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:25.181268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa07b-9a56-47ec-9aed-b426c9cb96e9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.199649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.202797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c5741-0d1e-4a80-a8dd-5cf5603ba1ca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.246223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.300192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.304822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa07b-9a56-47ec-9aed-b426c9cb96e9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.322313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 91/176] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:25.322313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.322313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.322313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.341715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:25.342936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:25.343861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.344786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.345076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.345359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] PATH_INFO: `/v3/users/2a598a92f6ea439a98503435cf969703` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.347717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a598a92f6ea439a98503435cf969703)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.349467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f846916-bf40-466f-878e-6a0ffa1e20f6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.351127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 86/177] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/users/2a598a92f6ea439a98503435cf969703 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:25.352441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5ff8-0dc5-4873-89c8-69b629611842 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.352741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5ff8-0dc5-4873-89c8-69b629611842 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.353024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a5ff8-0dc5-4873-89c8-69b629611842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.370240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d63a5ff8-0dc5-4873-89c8-69b629611842 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:25.371366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d63a5ff8-0dc5-4873-89c8-69b629611842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:25.421712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 92/178] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:25.422847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.452555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 87/179] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:25.453939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.454389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.454748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.474322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:25.475600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:25.502937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.546590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 88/180] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:25.547583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.559652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60017f-e5c6-41bc-b062-b388071c902c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.577714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.600717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.601394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.601720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.602014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.602901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:25.603223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.604725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.610739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 93/181] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:25.612146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.685397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7f31b-90a4-4bf4-abaa-974ab91393c0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.737644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac57b58-35e5-4d7c-8c3b-7a9ce8b8a947 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.738090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.739061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.739352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.739632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.741104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.742749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.744627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 89/182] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:25.745420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.770645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.789440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.789750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.790393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.790393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.863274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:25.871248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be6924-32a2-47d2-bf17-72ec136fd0db None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:25.886534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:25.886534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:25.913960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:25.914688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:25.915212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:25.915315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:25.926342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:25.934032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:25.935098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 90/183] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:25.936104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:25.998051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 94/184] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:25.998817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.023870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.045905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d64036-beeb-4b61-bd24-64f124c3d150 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.083668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.084825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.085119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.085403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.087159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.089096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.091191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 91/185] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.091795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.109175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1c66c-37de-4993-b66f-37cef676144e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.120281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.144982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.145831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.146115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.146391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.156513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:26.164790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:26.165711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:26.166521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 95/186] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:26.167241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.198928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceeef20-97cf-42d0-a6e8-b8ed0c1849db None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.233048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.233702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.233990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.234264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.235108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:26.235421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.237102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.242756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 92/187] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=5a2774adec8f4206a81f758406367b9f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.243534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.259568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.280138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a13afe-4ed4-4213-af66-550465c7eeb9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.319234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.320041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.320439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.320805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.321600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:26.321916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.323377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.334042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 96/188] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.339376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.351142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.364718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e003-df14-4901-bf56-609093a0f679 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.412097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.413040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.413341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.413604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] PATH_INFO: `/v3/projects/119df37b12e44827b73bf2cf2fe8a819` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.414809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] RBAC: Authorizing `identity:get_project(project_id=119df37b12e44827b73bf2cf2fe8a819)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.416273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844a72f-3976-449b-9b12-9f9f4725f1da None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.418026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 93/189] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/projects/119df37b12e44827b73bf2cf2fe8a819 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.419183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfd812-1f02-455d-a8d2-46ee7eddc176 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.419472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfd812-1f02-455d-a8d2-46ee7eddc176 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.419751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfd812-1f02-455d-a8d2-46ee7eddc176 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.422180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 94/190] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:26.422180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfd812-1f02-455d-a8d2-46ee7eddc176 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.462156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f9d5-7927-4037-ae82-938fb3921ee9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.499519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.500205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.500541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.500827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.501550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.503120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.534450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfd812-1f02-455d-a8d2-46ee7eddc176 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.568775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.569450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.569731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.570009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.570771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:26.571082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.572492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.581075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 95/191] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.581858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.587819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.599281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 97/192] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:26 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:26.599281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.607905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.703930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8b048-7786-415a-9c81-4c40ab5a3841 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.719795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc37c6-82ad-42ac-b512-a633c8e14701 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.745960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.746849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.747135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.747414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.762676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.764162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac124081-2368-4191-a50d-18a43d95eb92 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.765912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 96/193] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.767109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.767998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.768276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.770318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.771263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.771541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.771833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.772818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=119df37b12e44827b73bf2cf2fe8a819, user.id=2a598a92f6ea439a98503435cf969703) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:26.773145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.774639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.781151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 98/194] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/role_assignments?user.id=2a598a92f6ea439a98503435cf969703&scope.project.id=119df37b12e44827b73bf2cf2fe8a819&role.id=5a2774adec8f4206a81f758406367b9f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.782039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.787174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:26.788635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:26.863929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.884385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 97/195] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:26.885682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:26.897090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:26.930105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cee27-f451-4190-8d2e-45748314fb08 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:26.978536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:26.979358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:26.979647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:26.979934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:26.983389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:26.984930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:26.987270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 99/196] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:26.996437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.025320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16f538-656f-4a20-9011-353488fd92b8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.063570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.065309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.069226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.069520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.120420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367934ee-fc9a-475d-8621-b4bc56cc5ef5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.153634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.159853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.160982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.161494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.161971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.173716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:27.176914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.178431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:27.182668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:27.189637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 100/197] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:27.190456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.191158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.294924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447c16a-30a9-4f98-8ea9-3849ebe57a54 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.295281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 98/198] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:27.296094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.331823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.332880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.333381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.333669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.403222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aabae-d393-470e-9249-0cd6e589e035 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.404667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.412665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.413113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.414641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.446271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.446638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.452326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.452326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.452326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:27.452326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.452326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.466800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 99/199] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:27.467931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.550666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 101/200] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:27.551391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.605426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d8cd4-a554-4469-9c83-7255a018f014 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.647636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.648714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.649128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.649516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.650853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.652450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.654281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 100/201] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:27.655133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.678950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.691717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813746cf-0088-47cb-adca-8579decbdc30 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.696612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:27.730609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895927d6-4744-4742-a828-05535e183c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.731498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895927d6-4744-4742-a828-05535e183c21 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.731790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895927d6-4744-4742-a828-05535e183c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.732075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895927d6-4744-4742-a828-05535e183c21 None admin] PATH_INFO: `/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.733419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895927d6-4744-4742-a828-05535e183c21 None admin] RBAC: Authorizing `identity:get_user(user_id=9945ba10ce7b4d8797cc313f1dc3f7a7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.734969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895927d6-4744-4742-a828-05535e183c21 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.737186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 102/202] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/users/9945ba10ce7b4d8797cc313f1dc3f7a7 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:27.737989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895927d6-4744-4742-a828-05535e183c21 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.779694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23389d0a-9b5d-41a2-b6e7-386540e68425 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.815936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.816919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.817214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.817500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.818705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.820284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.822015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 101/203] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:27.822857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.906967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895927d6-4744-4742-a828-05535e183c21 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.954940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:27.955684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:27.955983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:27.956287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:27.957232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=3556159ccdd2403ca787dcc02864612c) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:27.957553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:27.959095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:27.970048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 103/204] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/role_assignments?user.id=3556159ccdd2403ca787dcc02864612c&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:27.971082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:27.988187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94976a-5088-4cd8-bf52-330dbfec40b6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:27.991856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:28.035162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:28.036149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:28.036557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:28.037056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:28.039900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:28.151039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:28.153017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:28.212549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b184370-7c0a-4838-825c-ae42120bf2b7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:28.251285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:28.252340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:28.252854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:28.253458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:28.254560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=3556159ccdd2403ca787dcc02864612c) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:28.255094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:28.256857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:28.262842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/role_assignments?user.id=3556159ccdd2403ca787dcc02864612c&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:28.359497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 102/206] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:28.365907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:28.500047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:28.518570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce4da6-7053-44c8-8dab-a21b88c0e866 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:28.570142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:28.571026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:28.571313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:28.571829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:28.573318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:28.575248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a3303-d76c-4949-95d0-26f3d1d923ea None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:28.577352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 105/207] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:28 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:28.585624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:28.593490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:28.771795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e14fb-1dcb-4a05-ad76-62cb912317df None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:28.824651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:28.825341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:28.825634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:28.825919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:28.826790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, scope.project.id=37c60ab16e7e4d378e715a01c3873f92, user.id=9945ba10ce7b4d8797cc313f1dc3f7a7) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:28.827109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:28.828693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8253f8d-9702-4945-9b48-93bbc9fb462d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:28.889420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 103/208] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:28 2022] GET /identity/v3/role_assignments?user.id=9945ba10ce7b4d8797cc313f1dc3f7a7&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:29.130851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca69c9-1ff5-49de-97ef-33e358527ae1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.131364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca69c9-1ff5-49de-97ef-33e358527ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.131364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca69c9-1ff5-49de-97ef-33e358527ae1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.132749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 106/209] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:29.139436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.139722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.140013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.148338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09897-043b-4af0-b83f-e05137799a67 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.148647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09897-043b-4af0-b83f-e05137799a67 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.148933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb09897-043b-4af0-b83f-e05137799a67 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.150454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 107/210] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:29.157431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.157723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.158669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.168603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:29.170201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:29.179127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:29.181546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:29.248405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:29.255390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 104/211] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:29.277738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 108/212] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:29.296252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:29.305015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:29.421504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:29.422773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb10f3-45b9-43bb-9e68-1d9f6dabcab8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:29.441757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8c4cb-6848-449b-98b3-7262471a1bf3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:29.486779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:29.487789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.488216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.488551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.490048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:29.492255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:29.495255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 105/213] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:29.500715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:29.502547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:29.503411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.503702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.504033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.515291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:29.523755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:29.524670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:29.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 109/214] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:29.530901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:29.575053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:29.624589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2c3a7-16f1-468a-a572-666d9b25b95c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:29.662292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28a2b-4b47-4f89-97d2-cb86bd85f770 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:29.668244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:29.669163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.669636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.670533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.671398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:29.674154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:29.701210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:29.701953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.702245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.702547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.704537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:29.704537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:29.705514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:29.714217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 110/215] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:29.719261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:29.731281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:29.788467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 106/216] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:29 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:05:29.833227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e90201e-5f5f-42a5-b98b-ef460e349efa None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:29.872447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:29.873394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:29.873687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:29.874046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] PATH_INFO: `/v3/users/59ad613f62b9457b8127705faafe9543` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:29.890692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] RBAC: Authorizing `identity:get_user(user_id=59ad613f62b9457b8127705faafe9543)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:29.892845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:29.894966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 111/217] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:29 2022] GET /identity/v3/users/59ad613f62b9457b8127705faafe9543 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:29.900675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.018881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ba968-f050-47d2-8de4-5618d8e605c8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:30.045387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:30.057651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:30.058604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.061602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.061602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.134164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:30.135657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:30.246537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 107/218] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:30.252000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.332562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:30.373698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f02-e3ee-43cb-b68a-86b28e353b15 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:30.410340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:30.411590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.412102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.412642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.414069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:30.415828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5570f-e11c-426e-bfe9-9c63b3fc3f45 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:30.417809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 112/219] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:30 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:30.424555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.467236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:30.513305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6f16-bbc6-45cf-ac60-e3ff84219f63 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.514005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6f16-bbc6-45cf-ac60-e3ff84219f63 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.514494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d6f16-bbc6-45cf-ac60-e3ff84219f63 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.516172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 113/220] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:30.530226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.531067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.531882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.565200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:30.566776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:30.573337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb12b93-a9f1-44c1-8cf4-3d688a3957d7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:30.609213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:30.609793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.610081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.610362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.612035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=59ad613f62b9457b8127705faafe9543) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:30.612378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:30.613850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd0045-4c12-4b64-8675-cc8dccb42c7c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:30.619994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 108/221] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:30 2022] GET /identity/v3/role_assignments?user.id=59ad613f62b9457b8127705faafe9543&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=5a2774adec8f4206a81f758406367b9f => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:30.629718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5bc84-c99a-4eb4-acab-30624a795e79 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.629909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5bc84-c99a-4eb4-acab-30624a795e79 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.630195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5bc84-c99a-4eb4-acab-30624a795e79 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.631607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 109/222] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:30.637424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.637745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.637993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.646128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 114/223] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:30.668869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.677483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:30.678752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:30.774424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 110/224] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:30.784067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:30.810347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.837543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:30.842120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748dc08-c7dc-4630-9b92-8203ec8540a6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:30.879813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:30.880729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:30.881171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:30.881553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:30.893887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:30.902088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:30.903163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:30.904147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:30.906151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 115/225] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:30.913468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:30.975118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e6026-7a76-42f7-8a16-e7ca6af29e84 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.012211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.013457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.013894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.014321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.026603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:31.036143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:31.037282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:31.038279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:31.040393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 111/226] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:31.046094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.130839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:31.155247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4221e3d-fb54-48e4-a2c6-9dffe17aade5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.190627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7266b-404c-46ff-a462-69c030f10b23 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.191723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.192604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.193059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.193490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.194416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:31.194885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.196481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.205952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 112/227] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:31.217384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.226550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:31.241121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.242239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.242529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.242808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.245078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:31.245078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.245314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.264991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 116/228] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:31.272290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.358811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f094cb-7638-484c-bbc9-7d7ffb5d95ae None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.414125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.415066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.415359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.415638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.416998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.418657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.424706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 113/229] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:31 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:31.426495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.436239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:31.489148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac17d59-88de-48fc-9956-7244c67598a3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.553873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.554762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.555046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.555325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.556735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.558373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.560331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 117/230] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:31 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:31.566982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.596762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3711cb-bc4d-4ea0-92f4-90d8ff77a122 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.631937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.632892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.633220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.633507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.634823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.636444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.638342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 114/231] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:31 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:31.644422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.703172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:31.724355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:31.750007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f20638-91fe-4953-a086-e6086be793e0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.785199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.786204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.786566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.786919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/3556159ccdd2403ca787dcc02864612c/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.789373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, user_id=3556159ccdd2403ca787dcc02864612c, role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.790862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8c56f-46e3-43be-bd04-8662e62bdfaa None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.800831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0b8d2-d16c-4a3c-adc3-8ddc2fb2ba22 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:31.837246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:31.838163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:31.838505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:31.838793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:31.840276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:31.841895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:31.843999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 118/232] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:31 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:31.851374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:31.876416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 115/233] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:31 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/3556159ccdd2403ca787dcc02864612c/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:31.984582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a75eb-32b6-44b6-8ebe-37caa3796d8b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:32.023551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:32.037840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:32.039871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:32.039871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:32.039871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/3556159ccdd2403ca787dcc02864612c/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:32.045257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, user_id=3556159ccdd2403ca787dcc02864612c, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:32.046790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362c281-470e-44dd-9419-bfa7426378d8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:32.176943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 119/234] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:31 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/3556159ccdd2403ca787dcc02864612c/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:32.644134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3acff-dace-459b-8e13-5f4ae67e22b8 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:32.644525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3acff-dace-459b-8e13-5f4ae67e22b8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:32.644728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3acff-dace-459b-8e13-5f4ae67e22b8 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:32.646317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 116/235] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:32.656842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:32.657456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:32.657456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:32.684501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:32.685791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:32.745313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1585-7673-4e5b-a271-e53a7320a3b8 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:32.745604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1585-7673-4e5b-a271-e53a7320a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:32.745800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1585-7673-4e5b-a271-e53a7320a3b8 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:32.747431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 117/236] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:32.752291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:32.752609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:32.752900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:32.778517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:32.779754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:32.806899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 120/237] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:32.853974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:32.890086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 118/238] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:32.946727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.016829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.042952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b708fa7-c913-40f0-a1d1-b1d5ecc0ef7a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.063023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.065786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3fe32-8db2-414b-9535-0f1e16161dea None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.090791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.092096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.092865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.093488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.095064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.097964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf5fc3-dc76-471e-8d29-27e3dfb5f09e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:33.101018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 121/239] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:33.102721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7daaa5-a117-4ab5-813c-e3121edf218a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.103171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7daaa5-a117-4ab5-813c-e3121edf218a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.109711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7daaa5-a117-4ab5-813c-e3121edf218a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.109711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 122/240] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:33.110718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7daaa5-a117-4ab5-813c-e3121edf218a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.123193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.124761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.125287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.126088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.140627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-82263239-f61a-4b22-bf03-6f0fdc794e20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:33.152840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:33.153762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:33.154571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:33.164468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 119/241] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:33.165875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.166167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.166447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.183915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:33.185142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:33.249764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7daaa5-a117-4ab5-813c-e3121edf218a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.263463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 120/242] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:33.263463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.313149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.313996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.314375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.314747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.315588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.317204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:33.336724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.383395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee138357-cd7b-4c5d-a5cb-71f1d8c6cba1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.418253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.419161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.419603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.420060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.420991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:33.421447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.423228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:33.431081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 123/243] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:33 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:33.433571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.435157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 121/244] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:33.441785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.480774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.575251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0312377-9b28-4de7-a95e-4cd9823f2771 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.630678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.631491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.631782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.632065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.642475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab6090-2f22-4754-b9b1-bd32dbbd4ef6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.642964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:33.650807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:33.651677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:33.652756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 124/245] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:33.660578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.661674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.680429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.681316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.681605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.681886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] PATH_INFO: `/v3/users/d3f52213e4984a65bbc2f94fd813658b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.696161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f52213e4984a65bbc2f94fd813658b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.697677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:33.700084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 122/246] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/users/d3f52213e4984a65bbc2f94fd813658b => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:33.709664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.788232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.796805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f09b9-6dfa-48c2-971e-050c71d57f63 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.840200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.841227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.841599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208abe5-50c1-4519-ad75-a3d5365b1ae6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:33.841989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.842551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.843831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:33.844386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.847107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:33.864373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 125/247] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:33.867182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:33.891353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:33.892059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:33.892360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:33.892645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:33.947087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:33.984776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:33.986882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.015304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b0734-5378-4207-9f7f-4f91c6fe0aa4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.064783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.066448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.067549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.068048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] PATH_INFO: `/v3/users/59ad613f62b9457b8127705faafe9543` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.069840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] RBAC: Authorizing `identity:get_user(user_id=59ad613f62b9457b8127705faafe9543)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:34.072757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.076423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 126/248] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/users/59ad613f62b9457b8127705faafe9543 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:34.082303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.126203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 123/249] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:34.127803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.128080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.205881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e4ac9-b949-4427-b575-4b9968ccfed0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.209258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.236030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2244da-5eef-45d8-911a-6d8caf309817 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.242697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.243571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.243858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.244138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.245573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:34.247732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.252778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 127/250] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:34.257462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.283904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.285390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.285884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.286386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.288024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:34.289845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.291907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 124/251] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:34.298060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.359920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.366962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f8a5d-4a5e-4fe8-9d3d-a34610824b7a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.404825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.406881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.407231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.407518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/59ad613f62b9457b8127705faafe9543/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.410469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, user_id=59ad613f62b9457b8127705faafe9543, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:34.411894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcf9519-8820-4cba-9588-63576e4e8f97 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.417853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688feda5-cde1-4839-a2fd-dc0dab5580f2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.456200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.457940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.458451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.459038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.460102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=d3f52213e4984a65bbc2f94fd813658b) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:34.460645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:34.462436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e674988-8995-4bb9-970a-d3bd1f2ccd99 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:34.474431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 125/252] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/role_assignments?user.id=d3f52213e4984a65bbc2f94fd813658b&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:34.476468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ecac4-bd85-4fee-92be-265a75c739e0 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.477047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ecac4-bd85-4fee-92be-265a75c739e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.477536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ecac4-bd85-4fee-92be-265a75c739e0 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.479530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 126/253] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:34.484874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.485486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.486119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.499061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.504645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 128/254] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:34 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/59ad613f62b9457b8127705faafe9543/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:05:34.515031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:34.516852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:34.609251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6cd1-080d-4404-b09f-b2d51a07ffc9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.609584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6cd1-080d-4404-b09f-b2d51a07ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.609818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c6cd1-080d-4404-b09f-b2d51a07ffc9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.612059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 129/255] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:34.618861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.619240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.619527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.623439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 127/256] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:34.657807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:34.659120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:34.675086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.764163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 130/257] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:34.810047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.869576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d583ce2-c3f7-4581-b4ff-e6bc0958fac1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.922630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.928133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.929646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.930103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.930552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.937435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:34.942196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11ad6e-4583-40af-a52b-77af82f43cbc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:34.957700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:34.967060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:34.968017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:34.976482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 128/258] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:34.978718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:34.981168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:34.981902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:34.982233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:34.982520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:34.993362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:35.002710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:35.003660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:35.004484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 131/259] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:35.024329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:35.158730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad746921-c1a8-40c8-b237-61b833a7e832 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.163005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e078b1-102f-42dc-bb10-aa0a70779314 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.194292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.194887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.195176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.195459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.196223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:35.196562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.197974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:35.207208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 132/260] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:35.218026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:35.225645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.226585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.227078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.227562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.228597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:35.229136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.230758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:35.237104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:35.247526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 129/261] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:35.253360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:35.376808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:35.410177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154e23e-5824-49b1-9b3e-4eebfc9298cf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.432964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3ee52-b8ce-40c3-b75d-5b788c5fe8e6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.500561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.501065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.502130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.502523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.502628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.502813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.502921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.503095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] PATH_INFO: `/v3/users/3556159ccdd2403ca787dcc02864612c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.504266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.505898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:35.507766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 130/262] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:35 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:35.508454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] RBAC: Authorizing `identity:get_user(user_id=3556159ccdd2403ca787dcc02864612c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.509936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:35.511939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 133/263] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:35 2022] GET /identity/v3/users/3556159ccdd2403ca787dcc02864612c => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:35.517473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:35.519728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:35.632447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0660ca-ebc5-4e09-be3b-521c982cf9a7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.660350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:35.660993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:35.678173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.678993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.679288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.686739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.701586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584b1bb-1572-4192-a367-97c66d4479a9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:35.758920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:35.759592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:35.759883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:35.760168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:35.827687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.829240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:35.887800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:35.889328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:36.021221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 131/264] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:36.026880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:36.070660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:36.079721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:36.127804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 134/265] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:36.134384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:36.268549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a2cf2-93f7-474f-aa45-450055060fd2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:36.295551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a1bc2-0f54-42ef-84fe-8d1e94f8cd85 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:36.335380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:36.336625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.337111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.337607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.339002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:36.340677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:36.342895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 132/266] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:36 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:36.349576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:36.350570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:36.351450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.351745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.352032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.358360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:36.359862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:36.362585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 135/267] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:36 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:36.368912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:36.433023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:36.504980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63020f26-4233-4137-ae71-01a59b58a578 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:36.557921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:36.558779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.559079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.559369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.560234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=3556159ccdd2403ca787dcc02864612c) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:36.560583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:36.562063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d6934-89ba-41ac-974e-63cbd869fbfa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:36.570322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 136/268] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:36 2022] GET /identity/v3/role_assignments?user.id=3556159ccdd2403ca787dcc02864612c&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:36.576088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:36.594830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd11158-5ba2-4863-a6b4-809c6a9fceaa None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:36.644139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b679-1628-4cf3-936b-e4814cd35462 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.645077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b679-1628-4cf3-936b-e4814cd35462 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.645976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317b679-1628-4cf3-936b-e4814cd35462 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.652394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 137/269] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:36.654676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.655284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.655772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.662620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:36.663302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.663588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.663868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.664944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=3556159ccdd2403ca787dcc02864612c) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:36.665259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:36.666741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:36.679698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 133/270] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:36 2022] GET /identity/v3/role_assignments?user.id=3556159ccdd2403ca787dcc02864612c&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:36.684590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:36.685853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:36.783843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 138/271] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:36.809150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:36.943865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a1155-2f47-4de6-8164-d47f41ca496a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:36.980845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:36.981650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:36.981941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:36.982214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:36.995129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:37.002925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:37.003842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:37.005101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 134/272] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:37.010054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.049386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.118665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faf1e7-320b-4272-9540-af1ca8bb2799 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:37.156836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:37.157520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.158108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.158264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.158917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:37.159226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:37.160668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f6020f-5a05-4410-bd84-fdd8da88f90f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:37.171300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 139/273] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:37.177444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.199220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.291448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5060-4f3b-46ca-bc28-4feb7d4f5e8c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.291930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5060-4f3b-46ca-bc28-4feb7d4f5e8c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.292333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae5060-4f3b-46ca-bc28-4feb7d4f5e8c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.294171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 140/274] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:37.296733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e06d4-4815-4750-8764-86455a4411e9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:37.302043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.302338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.302623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.320375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:37.321587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:37.345006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:37.346739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.347024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.347308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] PATH_INFO: `/v3/users/4f748ac47132416280d30eede2c5ebd9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.364086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] RBAC: Authorizing `identity:get_user(user_id=4f748ac47132416280d30eede2c5ebd9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:37.366134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:37.368533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 135/275] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/users/4f748ac47132416280d30eede2c5ebd9 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:37.374000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.409994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 141/276] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:37.436434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.457754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.478654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.481401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0f603-db71-49cc-8f80-e01b86b14d1e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:37.516897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:37.518319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.518602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.518885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.519730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, user.id=4f748ac47132416280d30eede2c5ebd9, scope.system=all) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:37.520043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:37.521545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:37.526811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 136/277] 10.222.0.47 () {56 vars in 1308 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/role_assignments?user.id=4f748ac47132416280d30eede2c5ebd9&scope.system=all&role.id=5a2774adec8f4206a81f758406367b9f => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:37.566089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990334c-9935-4ed4-b52a-5c975c9e1c86 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:37.607043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:37.608260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.608792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.609280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.611355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:37.613164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b527d0c-8914-4e07-8ccc-7a520b723fcf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:37.615186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 142/278] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:37.620909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.694849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899135-ebcc-4fef-8cd4-3e4076253f9d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.695376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899135-ebcc-4fef-8cd4-3e4076253f9d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.695682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899135-ebcc-4fef-8cd4-3e4076253f9d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.697223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 143/279] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:37.699216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52006f49-2e25-4f35-a894-deee1b005951 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.699507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52006f49-2e25-4f35-a894-deee1b005951 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.699789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52006f49-2e25-4f35-a894-deee1b005951 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.701047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 144/280] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:37.705320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bcd8-aa49-4a8e-874f-0583f72df456 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.705817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bcd8-aa49-4a8e-874f-0583f72df456 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.706108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808bcd8-aa49-4a8e-874f-0583f72df456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.732375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f808bcd8-aa49-4a8e-874f-0583f72df456 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:37.733663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f808bcd8-aa49-4a8e-874f-0583f72df456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:37.742144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcd2aa-8f66-4882-903e-9c8cf78ad6ad None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:37.778605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:37.779282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.779566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.779851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.780587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:37.782191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:37.793012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.807080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 145/281] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:37.808495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:37.808761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:37.809046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:37.826167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:37.827361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:37.849408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:37.896344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 137/282] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:05:37 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:37.896344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:37.906359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 146/283] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:37.930391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.007341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956b8f5-2771-47a5-bbb7-ebf76687a839 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.049221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00da4a1-408f-4a9b-b2ef-9de633aeb7ee None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.056942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.057876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.058232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.058581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.075652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:38.084542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:38.085471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:38.086282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:38.090631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 138/284] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:38.091809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.091945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.092654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.092943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.093224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.106532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:38.114949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:38.115866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:38.116720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 147/285] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:38.123340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.149438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.161493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.227900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f45ab-d4c8-487f-b499-4b587a6a7f32 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.265339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5066f-ca9a-445b-a319-e56d7d3b4997 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.268702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.269401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.269689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.269977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.270757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:38.271073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.272943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.288412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 139/286] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:38.290743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.312494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.313324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.313702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.314162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.315162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:38.315586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.317487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.330047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 148/287] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:38.335438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.417077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc4a1-e871-445e-9210-8243e1a36533 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.436967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.450165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.457056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.458129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.458560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.458983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] PATH_INFO: `/v3/users/59ad613f62b9457b8127705faafe9543` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.460378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] RBAC: Authorizing `identity:get_user(user_id=59ad613f62b9457b8127705faafe9543)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.462014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.463855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 140/288] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/users/59ad613f62b9457b8127705faafe9543 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:38.469475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.476531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bc11c-deaf-48c1-81f9-adc825ccb356 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.515843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.520539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.520539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.520539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] PATH_INFO: `/v3/users/d3f52213e4984a65bbc2f94fd813658b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.520539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] RBAC: Authorizing `identity:get_user(user_id=d3f52213e4984a65bbc2f94fd813658b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.520539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.522607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 149/289] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/users/d3f52213e4984a65bbc2f94fd813658b => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:38.527593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.621493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717f740-f15c-431c-85be-4fb064bc5d6a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.662046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.662709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.663002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.663283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.756013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f715350-d454-4124-a318-6c3f8480332e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:38.756522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.758024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.776738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.818516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:38.820117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:38.820686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:38.821486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:38.829457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:38.831499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:38.834241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 150/290] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:38.839662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:38.868874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 141/291] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 18:05:38.873472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:38.878286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:39.004690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:39.013656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b92f1-1a4b-440b-b682-4569a9bb8143 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:39.079452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:39.081827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:39.084546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:39.085034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:39.087249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:39.090041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:39.095114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 142/292] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:38 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:39.102402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89fde3-d779-4b0b-93b1-1375a312ee49 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:39.117682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:39.194039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:39.194963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:39.195259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:39.195544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/d3f52213e4984a65bbc2f94fd813658b/roles/d54be0042c4c4a2fb1380e52eb16b570` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:39.201174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, user_id=d3f52213e4984a65bbc2f94fd813658b, role_id=d54be0042c4c4a2fb1380e52eb16b570)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:39.201174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f207545d-7b93-4c50-8c42-a900ad1b7838 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:39.246396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced2338-ac93-4848-8c82-c36c94549fa4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:39.284745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:39.285422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:39.285710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:39.285998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:39.286849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=59ad613f62b9457b8127705faafe9543) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:39.287165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:39.288735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21147b-58b2-4c22-a3a7-4882ea303733 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:39.297530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 151/293] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:05:38 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/users/d3f52213e4984a65bbc2f94fd813658b/roles/d54be0042c4c4a2fb1380e52eb16b570 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:39.299274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 143/294] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:39 2022] GET /identity/v3/role_assignments?user.id=59ad613f62b9457b8127705faafe9543&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=5a2774adec8f4206a81f758406367b9f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:39.818065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd112d-a993-487e-a5ce-f279f7cb1e69 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:39.818403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd112d-a993-487e-a5ce-f279f7cb1e69 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:39.818619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd112d-a993-487e-a5ce-f279f7cb1e69 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:39.820061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 152/295] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:39.825678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:39.825973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:39.826251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:39.847782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:39.849334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:39.864143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:39.926108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 144/296] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:39.952485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd112d-a993-487e-a5ce-f279f7cb1e69 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.035265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:40.061990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd112d-a993-487e-a5ce-f279f7cb1e69 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.098122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.099097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.099380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.099657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.110182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:40.118399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:40.119329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:40.120104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 153/297] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:40.127566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.232494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed4f12-de04-491f-aa76-877522965fe6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.266806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.268535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.268848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.269117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.269880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:40.270188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:40.271565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6781c2-8ab3-4382-80b7-ec91eea5b530 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:40.280321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 145/298] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:40.289429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.374774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9306d8-b9f5-4c8b-905b-dbcea0c68580 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.375491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9306d8-b9f5-4c8b-905b-dbcea0c68580 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.375995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9306d8-b9f5-4c8b-905b-dbcea0c68580 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.378304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 146/299] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:40.383631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.384174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.384680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.402684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5413a5-c5fb-49b2-bae6-d54525c0a351 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.406912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:40.408878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:40.412616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:40.446814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.449106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.449593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.450111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] PATH_INFO: `/v3/users/4f748ac47132416280d30eede2c5ebd9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.453122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] RBAC: Authorizing `identity:get_user(user_id=4f748ac47132416280d30eede2c5ebd9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:40.454707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:40.461471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 154/300] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/users/4f748ac47132416280d30eede2c5ebd9 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:40.462307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.465041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:40.494277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 147/301] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:40.519151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.581414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589fe9-4f05-4af2-80b8-566958571669 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.620936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.621901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.622273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.622642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] PATH_INFO: `/v3/system/users/4f748ac47132416280d30eede2c5ebd9/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.624221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4f748ac47132416280d30eede2c5ebd9, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:40.625769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b275f-d017-47fa-97c9-3ebf5871b87a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:40.637831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99f9ae-d033-442c-b406-50a5f99a2618 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.674396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.675245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.675532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.675806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.683218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:40.686107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-050acdc9-31dc-4717-862a-bb6dec97c7a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:40.694133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:40.695475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:40.696649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 148/302] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:40.697312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c032532-6017-4ce0-8f69-41270c1a8cd1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.697312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c032532-6017-4ce0-8f69-41270c1a8cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.697312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c032532-6017-4ce0-8f69-41270c1a8cd1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.698720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 149/303] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:40.701219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c032532-6017-4ce0-8f69-41270c1a8cd1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.727279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 155/304] 10.222.0.47 () {58 vars in 1334 bytes} [Tue Jul 12 18:05:40 2022] PUT /identity/v3/system/users/4f748ac47132416280d30eede2c5ebd9/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:40.727279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.727279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.727279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.758732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:40.760384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:40.835975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:40.837647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c032532-6017-4ce0-8f69-41270c1a8cd1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:40.843970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 156/305] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:40.869829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.878642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:40.878822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:40.879005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:40.879282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:40.880054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:40.881327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:40.881913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:40.892392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 150/306] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:40.898027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:40.968781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:41.000208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe846d-9b37-4aa4-a538-b4c74966c152 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.009254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d9c9-7a49-45ac-bc53-0e8d847f32a6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.037901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.038861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.039249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.039628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.040932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:41.042497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:41.044579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 157/307] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:41.048962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.050014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:41.051625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.052012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.052433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] PATH_INFO: `/v3/users/a072468857984af2929db44afd2a2359` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.071385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] RBAC: Authorizing `identity:get_user(user_id=a072468857984af2929db44afd2a2359)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:41.073348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:41.075990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 151/308] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:40 2022] GET /identity/v3/users/a072468857984af2929db44afd2a2359 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:41.083320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:41.128202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:41.169454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91af9b-0195-441e-afa3-5a4e35491762 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.212279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf4047-b285-42cb-9958-29cc9f96412a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.214495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.216136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.216631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.217094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] PATH_INFO: `/v3/groups` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.218172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:41.219804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad3c46-0e2e-477a-93f3-105e7c678e8d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:41.235427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:41.249263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.250000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.250359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.250843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.252586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, user.id=a072468857984af2929db44afd2a2359, scope.system=all) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:41.252981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:41.254693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b40fbe-ff2b-4bec-a67c-9bf5f0ed73cf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:41.260034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 152/309] 10.222.0.47 () {56 vars in 1308 bytes} [Tue Jul 12 18:05:41 2022] GET /identity/v3/role_assignments?user.id=a072468857984af2929db44afd2a2359&scope.system=all&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:41.295340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 158/310] 10.222.0.47 () {60 vars in 1134 bytes} [Tue Jul 12 18:05:41 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:41.523959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f08-593d-4cfe-bf44-9393ce8f9aab None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.524487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f08-593d-4cfe-bf44-9393ce8f9aab None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.524590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3130f08-593d-4cfe-bf44-9393ce8f9aab None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.525940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 153/311] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:41.531416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.531987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.532538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.550482 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:41.551990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:41.628405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 159/312] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:41.648216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3130f08-593d-4cfe-bf44-9393ce8f9aab None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:41.754565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3130f08-593d-4cfe-bf44-9393ce8f9aab None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.790596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.801545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.801545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.801545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.804050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:41.812143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:41.813272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:41.814599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:41.821524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 154/313] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:41.823718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:41.887537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:41.934100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ac594-8a98-4332-96fa-50df69ea1c97 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:41.970768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:41.971477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:41.971773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:41.972056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:41.972859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:41.973414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:41.974828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:41.984230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 160/314] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:41.990686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:42.039941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:42.098605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03084090-8222-4427-b4af-5675b67a8c99 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:42.133788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:42.134956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.135437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.135928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] PATH_INFO: `/v3/users/d3f52213e4984a65bbc2f94fd813658b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.137360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] RBAC: Authorizing `identity:get_user(user_id=d3f52213e4984a65bbc2f94fd813658b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:42.138993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:42.145916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 155/315] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:41 2022] GET /identity/v3/users/d3f52213e4984a65bbc2f94fd813658b => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:42.147462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:42.250087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b605d1-03d8-48c2-9982-d7c785183bf4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:42.268027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:42.287830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:42.288829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.289341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.289816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.359181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:42.361183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:42.464176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 161/316] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:42.474149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:42.580746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e056e-d46d-479b-8c52-31b00c83ff84 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:42.596678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:42.616161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:42.617342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.617824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.618297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.619752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:42.621459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d9e2c-4222-4ef2-a2aa-622e2e5a8aca None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:42.628574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 156/317] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:42 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:42.633502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:42.651964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:42.738525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8760c8-99ff-479a-b224-80b02ee320db None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:42.774768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:42.775433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.775725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.776015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.776944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855, user.id=d3f52213e4984a65bbc2f94fd813658b) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:42.777268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:42.778772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0106b-36d1-48fa-9167-2827a242083c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:42.785235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 162/318] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:05:42 2022] GET /identity/v3/role_assignments?user.id=d3f52213e4984a65bbc2f94fd813658b&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:42.843053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4193-57c8-44ea-80db-2fde78b2c0c2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.843736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4193-57c8-44ea-80db-2fde78b2c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.844215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4193-57c8-44ea-80db-2fde78b2c0c2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.846471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 157/319] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:42.852610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:42.852899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:42.853182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:42.894049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:42.895266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:42.975456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 163/320] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:43.001064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4193-57c8-44ea-80db-2fde78b2c0c2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.118275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4193-57c8-44ea-80db-2fde78b2c0c2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.152868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.153984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.154464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.154953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.165914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:43.174610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:43.175473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:43.176261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:43.182236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 158/321] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:43.183760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.196763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.286609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ef7f6-0e1d-4ab2-b515-fe16d125cf44 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.321132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.322144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.322652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.323125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.324093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:43.324768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:43.326356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab6f76-536f-4b34-bcca-e08614418529 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:43.340003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 164/322] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:43.341727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.385564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.409123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff11d-5383-472b-a138-d6af518c7d27 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.409401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff11d-5383-472b-a138-d6af518c7d27 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.409604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fff11d-5383-472b-a138-d6af518c7d27 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.411104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 165/323] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:43.416127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b007910-3671-47ea-ac54-e808026d02c7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.416451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b007910-3671-47ea-ac54-e808026d02c7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.416736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b007910-3671-47ea-ac54-e808026d02c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.434241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2b007910-3671-47ea-ac54-e808026d02c7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:43.435610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2b007910-3671-47ea-ac54-e808026d02c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:43.454233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8f291-157c-4f61-a9e6-8cca57dedeac None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.489585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.490470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.490758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.491039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] PATH_INFO: `/v3/users/4f748ac47132416280d30eede2c5ebd9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.492248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] RBAC: Authorizing `identity:get_user(user_id=4f748ac47132416280d30eede2c5ebd9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:43.493763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5b0b-1f85-4339-8041-a1d786b26f6a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:43.495652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 159/324] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/users/4f748ac47132416280d30eede2c5ebd9 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:43.496749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770632ee-a6ee-40fc-9914-48a7e60ce6e2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.497039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770632ee-a6ee-40fc-9914-48a7e60ce6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.497319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770632ee-a6ee-40fc-9914-48a7e60ce6e2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.498541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 160/325] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:43.501046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770632ee-a6ee-40fc-9914-48a7e60ce6e2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.507915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 166/326] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:43.509278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.509669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.509904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.514942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.526675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:43.527876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:43.596182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 167/327] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:43.597054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.605653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770632ee-a6ee-40fc-9914-48a7e60ce6e2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.617051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.641565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.643033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.643382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.643678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.644560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] RBAC: Adding query filter params (role.id=5a2774adec8f4206a81f758406367b9f, user.id=4f748ac47132416280d30eede2c5ebd9, scope.system=all) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:43.644878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:43.646678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:43.662317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 161/328] 10.222.0.47 () {56 vars in 1308 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/role_assignments?user.id=4f748ac47132416280d30eede2c5ebd9&scope.system=all&role.id=5a2774adec8f4206a81f758406367b9f => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:43.663159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.712393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508eec6-c2d7-49f8-b0de-4316986e8190 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.745028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.750167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.757948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.757948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.757948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.757948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:43.757948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:43.760598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 168/329] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:43.761774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.785172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e4499-183e-4d32-a721-a9f0af4b2575 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.825351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.826161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.826436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.826718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.838930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:43.849949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:43.851052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:43.851854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 162/330] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:43.857124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:43.897200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:43.903509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d95c3-5615-4349-92fd-c2adacb2eeb6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:43.936840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:43.937428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:43.937715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:43.937993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] PATH_INFO: `/v3/groups` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:43.938688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:43.940153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:43.971738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ddb75-9ff0-425b-a649-facdd03dde2f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:44.006385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:44.007057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:44.007349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:44.007634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:44.008425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:44.008740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:44.010142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:44.019268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 163/331] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:44.024660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:44.043312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 169/332] 10.222.0.47 () {60 vars in 1134 bytes} [Tue Jul 12 18:05:43 2022] POST /identity/v3/groups => generated 226 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:05:44.118278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:44.131055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f511e-c12d-47a5-ba6c-55d34c36b5ef None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:44.171598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:44.172513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:44.172803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:44.173248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] PATH_INFO: `/v3/users/a072468857984af2929db44afd2a2359` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:44.174490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=a072468857984af2929db44afd2a2359)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:44.175973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7589179-c1df-4bf0-b320-fed1fef59fe6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:44.177655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 164/333] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:44 2022] GET /identity/v3/users/a072468857984af2929db44afd2a2359 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:44.183338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:44.247107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:44.295909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb12c5-fe06-4de4-a4de-87881da1b87d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:44.334681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:44.335540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:44.335821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:44.336098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] PATH_INFO: `/v3/system/users/a072468857984af2929db44afd2a2359/roles/d54be0042c4c4a2fb1380e52eb16b570` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:44.337640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a072468857984af2929db44afd2a2359, role_id=d54be0042c4c4a2fb1380e52eb16b570)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:44.339028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60b873-e108-4669-b8ba-3fd6ef873ecb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:44.415643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 170/334] 10.222.0.47 () {58 vars in 1334 bytes} [Tue Jul 12 18:05:44 2022] PUT /identity/v3/system/users/a072468857984af2929db44afd2a2359/roles/d54be0042c4c4a2fb1380e52eb16b570 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:46.606776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d811-d603-4db9-967c-a58ef17ef152 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.607766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d811-d603-4db9-967c-a58ef17ef152 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.608254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286d811-d603-4db9-967c-a58ef17ef152 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.610171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 165/335] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:46.621881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.622800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.623278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.644972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:46.646468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:46.726627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 171/336] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:46.741265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8598f-6643-4d76-8267-120b77c2a9ff None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.741853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8598f-6643-4d76-8267-120b77c2a9ff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.742324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8598f-6643-4d76-8267-120b77c2a9ff None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.743881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 166/337] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:46.753547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:46.755574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67f0ff-c44d-489c-b62a-3d39a8d1eeac None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.755876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67f0ff-c44d-489c-b62a-3d39a8d1eeac None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.756164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67f0ff-c44d-489c-b62a-3d39a8d1eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.778100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3b67f0ff-c44d-489c-b62a-3d39a8d1eeac None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:46.781255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3b67f0ff-c44d-489c-b62a-3d39a8d1eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:46.841772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:46.913752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 167/338] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:46.915560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214f01-3826-4db6-bf1e-e25fecefdc05 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.916328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214f01-3826-4db6-bf1e-e25fecefdc05 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.918136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b214f01-3826-4db6-bf1e-e25fecefdc05 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.920946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 168/339] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:46.925519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:46.927272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:46.928497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:46.934475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab099880-626f-4d90-824e-68cc9a806e56 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:46.970044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:46.973395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:47.011357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.012712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.014061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.017740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.037072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:47.053917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:47.054893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:47.055692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:47.062530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 172/340] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:47.064061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.120214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 169/341] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:47.122361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.154747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.264797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57ccaa-3fe3-4ad7-b031-12e3b45a7d32 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.286203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.312276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.313116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.313412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.313927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.324412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e6fd-c256-4f52-8dfe-52df27983b76 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.327133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:47.338082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:47.338980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:47.340222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 173/342] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:47.340896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.400042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.400834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.401130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.401483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.402370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:47.402745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:47.404436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:47.414985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 170/343] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:47.416177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.471257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3aa21-d0d4-4b8a-93e4-b0061ca1984e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.514539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.515410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.515644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.515926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.528916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.532884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b73e7bf4-64cf-4dee-ab77-d1a63b66b7b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:47.542693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:47.546724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:47.547617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 174/344] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:47.550013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327938-ea96-442d-91a2-4a24cae2bfa3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.550311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327938-ea96-442d-91a2-4a24cae2bfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.550594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4327938-ea96-442d-91a2-4a24cae2bfa3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.551958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 175/345] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:47.552744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4327938-ea96-442d-91a2-4a24cae2bfa3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.557675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207d724-5c7e-4f6d-b896-698afad55e76 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.597418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.598783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.600161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.600161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.600161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.600803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:47.601111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:47.602509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d3bb4-23b5-433e-8c56-72b788bde981 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:47.612730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 171/346] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:47.614054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.614349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.614631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.631193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:47.632444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:47.688740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4327938-ea96-442d-91a2-4a24cae2bfa3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.718634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 172/347] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:47.720679 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.743041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.743712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.743999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.744279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.817056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.834689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:47.836509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:47.853885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825642f-3e92-45fd-8c3f-48cbca3712a1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:47.872891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:47.894489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:47.895526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:47.896027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:47.896547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:47.897569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:47.898074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:47.899835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:47.909824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 173/348] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:47.910978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:47.948578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 176/349] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:47.949453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.027128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7252201-d9b3-4c65-802e-f25148885613 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.057828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73f7a-726e-46f6-a4d9-90479dc25738 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.060528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.068506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.068506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.068881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.069107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] PATH_INFO: `/v3/users/a072468857984af2929db44afd2a2359` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.070286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] RBAC: Authorizing `identity:get_user(user_id=a072468857984af2929db44afd2a2359)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.071742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.073545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 174/350] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/users/a072468857984af2929db44afd2a2359 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.076351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.097444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.098512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.098797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.099074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.102257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.112299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:48.121432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:48.122495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 177/351] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:48.123472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.209311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159198b-4f85-4fc7-9775-6d33e6986c04 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.251755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.252452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.252738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.253022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.259697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df172c-754a-417e-86e5-84b7172150c6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.301143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.302016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.302299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.302577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.303771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.305324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0def7d-1405-4e19-8daf-abc0b613886a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.307394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 178/352] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.310589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f204e-9e44-49e6-84fe-ed81a9a42072 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.310589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f204e-9e44-49e6-84fe-ed81a9a42072 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.310589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f204e-9e44-49e6-84fe-ed81a9a42072 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.315487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 179/353] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:48.316387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f204e-9e44-49e6-84fe-ed81a9a42072 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.352579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.358284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.359770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.405744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.462114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f204e-9e44-49e6-84fe-ed81a9a42072 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.504037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 175/354] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:48.504945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.507991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.508986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.509531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.509531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.510566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] RBAC: Adding query filter params (role.id=d54be0042c4c4a2fb1380e52eb16b570, user.id=a072468857984af2929db44afd2a2359, scope.system=all) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:48.510952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.512606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf0f3f-7328-41f6-8f4c-872727369890 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.527559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 180/355] 10.222.0.47 () {56 vars in 1308 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/role_assignments?user.id=a072468857984af2929db44afd2a2359&scope.system=all&role.id=d54be0042c4c4a2fb1380e52eb16b570 => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.532791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.532791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.532791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.549692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:48.551134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:48.647395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.652253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58defe5c-015c-484c-8c3d-770ff6ce5973 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.659669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 181/356] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:48.663922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.706022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.706993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.706993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.707174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.708894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:48.708894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.709568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.719206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 176/357] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.720868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.729684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.795223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa18b08-f10b-4b6c-bc09-6054ffd27dcc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.837734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.839149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.839612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.839892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.840866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df071612-07e8-42c6-83c6-2d0bce60d5f0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.847010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.848795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5d947-d7d0-44d3-a271-b692bd7bc4c7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.850868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 182/358] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.851867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ffa79-d641-43af-87d4-d47c53c1311a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.852157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ffa79-d641-43af-87d4-d47c53c1311a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.852470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ffa79-d641-43af-87d4-d47c53c1311a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.853724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 183/359] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:48.854481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ffa79-d641-43af-87d4-d47c53c1311a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.884700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:48.885572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:48.885866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:48.886149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:48.887373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:48.888883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:48.890590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 177/360] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:48.891360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:48.928075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:48.976503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ffa79-d641-43af-87d4-d47c53c1311a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:48.994312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.017480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.017965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfb55d-26d5-4b40-ac61-a13d7d9ca5f9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.018301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.020433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.020433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.032780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:49.041205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:49.042424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:49.043373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:49.045894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 184/361] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:49.046789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.055592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.056259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.056587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.056999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.137336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.139212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.169842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee1a9d-63a1-48a4-af57-f437a5b1b1bf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.170646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.210825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.211424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.211727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.212009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.212911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=37c60ab16e7e4d378e715a01c3873f92) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:49.213236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.214694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78a319-94ed-4a67-94b2-c8bfdffee093 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.221007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 185/362] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:49.222227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.222524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.222814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.242673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:49.244653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:49.261847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 178/363] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:49.266931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.288803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.338679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 186/364] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:49.340230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.404609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4661dfa-c135-40ea-9a45-a0e6e93871c6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.450660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.458964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.466390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 179/365] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:49.471523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.483382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a96eba-6bd3-4ce3-bc4b-56610f540781 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.524354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.525040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.525332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.525616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.526402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:49.526697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.528109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.538104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 187/366] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:49.538797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.582391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.590598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc857a4f-ca00-4b3e-b887-d9b846fa7f8e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.645649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.647297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.647600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.648144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.650810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.652683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.654620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 180/367] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:49.655405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.680217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5dd25e-ad25-4966-a9ae-2403962657ee None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.726325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.726325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.726325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.726325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.740006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:49.741026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:49.749260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:49.755057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 188/368] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:49.755840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.789253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cc86f-a0d0-4757-a26d-1c8a6f16b926 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.832011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.832739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.833033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.833318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.834172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=37c60ab16e7e4d378e715a01c3873f92) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:49.834496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:49.835961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:49.842036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 181/369] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=5a2774adec8f4206a81f758406367b9f => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:49.842942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:49.875321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:49.888664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4a870-9130-4b5f-b3eb-90d78bc1b488 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:49.939647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74927136-e3a9-4859-a956-02b821049505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:49.940457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927136-e3a9-4859-a956-02b821049505 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:49.940756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927136-e3a9-4859-a956-02b821049505 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:49.941099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74927136-e3a9-4859-a956-02b821049505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:49.969397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397278d-c7be-4119-9c4f-52b9c7d197f6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.023433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.030150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.031298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.031590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.031873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.033453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.044831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.044831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 182/370] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.044831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.058898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74927136-e3a9-4859-a956-02b821049505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.060838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74927136-e3a9-4859-a956-02b821049505 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.148484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8819de9-cf03-4abf-9a9e-8f2a6973208d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.164247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.176562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 189/371] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:50.177782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74927136-e3a9-4859-a956-02b821049505 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.185867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.187028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.187028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.187221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.187975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:50.188259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.190121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.199101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 183/372] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.199995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.268255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.288133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74927136-e3a9-4859-a956-02b821049505 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.315428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08e71c-9980-4c51-b37b-856d3ac74064 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.326614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.327591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.328095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.328618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.330190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:50.330798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.332516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.338839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 190/373] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=5a2774adec8f4206a81f758406367b9f => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.339931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.354853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.355148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.355425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.356708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.358219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.361196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 184/374] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.366933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.414439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.472266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da1b9d-8f19-460c-b48d-03b6091d4489 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.508951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f643-128d-4a05-be75-faa96beb8610 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.524352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.525264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.525675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.526392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.576478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.605689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.606570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.607323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.607323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.608337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.609803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.610043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.612557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 185/375] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.613465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.617837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.676730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:50.726480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 191/376] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:50.733780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:50.748768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1ad13-c4d1-4db1-8560-6306a0e823ee None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.785382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.786054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.786342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.786625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.787476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:50.787790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.789404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.800124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 186/377] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:50.910813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65e85e-d6a0-4ba8-a57d-2e0a20adf728 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:50.955595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:50.956508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:50.956799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:50.957084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] PATH_INFO: `/v3/projects/cecdefe3d592442ba46417c3dd5f54f9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:50.958529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] RBAC: Authorizing `identity:get_project(project_id=cecdefe3d592442ba46417c3dd5f54f9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:50.960044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:50.962123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 192/378] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/projects/cecdefe3d592442ba46417c3dd5f54f9 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:50.967235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:51.034889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:51.076801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331026-2b89-4310-bb51-3bfe5cf35088 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:51.112778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:51.113651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:51.113945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:51.114228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] PATH_INFO: `/v3/groups/decaf77e9dfd461f9162a3c08c69f72d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:51.120719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] RBAC: Authorizing `identity:get_group(group_id=decaf77e9dfd461f9162a3c08c69f72d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:51.122202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231dc72-74c8-4cb4-94c5-e72691560847 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:51.124219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 187/379] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:50 2022] GET /identity/v3/groups/decaf77e9dfd461f9162a3c08c69f72d => generated 229 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:51.129878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:51.168760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:51.255313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b2bf9-5de0-443a-8b40-859eff605720 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:51.296645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:51.297385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:51.297789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:51.298110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:51.298994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] RBAC: Adding query filter params (group.id=decaf77e9dfd461f9162a3c08c69f72d, role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=cecdefe3d592442ba46417c3dd5f54f9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:51.299319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:51.300947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a946b-bb4e-4257-a5db-2794e6690d17 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:51.307912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 193/380] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:51 2022] GET /identity/v3/role_assignments?group.id=decaf77e9dfd461f9162a3c08c69f72d&scope.project.id=cecdefe3d592442ba46417c3dd5f54f9&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:51.630267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9d169-1913-4ed6-a5d0-1c3d90f96b44 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:51.630556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9d169-1913-4ed6-a5d0-1c3d90f96b44 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:51.630844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9d169-1913-4ed6-a5d0-1c3d90f96b44 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:51.632479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 188/381] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:51.637829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:51.638124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:51.638401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:51.655243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:51.656473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:51.749804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 194/382] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:51.776332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9d169-1913-4ed6-a5d0-1c3d90f96b44 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:51.884922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9d169-1913-4ed6-a5d0-1c3d90f96b44 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:51.920944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:51.922029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:51.922500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:51.923458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:51.923909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:51.934700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:51.943863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:51.944986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:51.945966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:51.948336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 189/383] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:51.956359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:51.988715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:52.099429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0102a8-245f-4eb0-a259-cc4277c7f460 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:52.135949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:52.136676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.136970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.137256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.138030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:52.138346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:52.139765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:52.148883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 195/384] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:52.154342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:52.300200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbde572-22f7-4f3f-9057-504b1fa67e17 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:52.339717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:52.340971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.341500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.341984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.343394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:52.345176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:52.347349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 190/385] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:52 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:52.357548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:52.464731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:52.496485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167dd51-5abc-431c-ad5b-929b87ea5af3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:52.545540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:52.546426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.546719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.547002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.548898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:52.550377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c70b-8cd7-44b6-aaa2-2ddc4df09388 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:52.552436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 196/386] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:52 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:52.557710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:52.634080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:52.664117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0307e38-09d8-4448-b4f2-40374173f714 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.664454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0307e38-09d8-4448-b4f2-40374173f714 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.664745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0307e38-09d8-4448-b4f2-40374173f714 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.666386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 197/387] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:52.671224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.671522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.671810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.700190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a801795c-78d5-4b7c-b62e-72c7e47a621f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:52.713883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:52.715084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:52.757605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:52.758501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.758793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.759075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/groups/b3f3503d679c4b70967b566b18280044/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.761315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, group_id=b3f3503d679c4b70967b566b18280044, role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:52.763151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb9a29d-845a-4508-871d-243433af6ad4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:52.818433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 198/388] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:52.841570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:52.850199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 191/389] 10.222.0.47 () {58 vars in 1442 bytes} [Tue Jul 12 18:05:52 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/groups/b3f3503d679c4b70967b566b18280044/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:52.918240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f06e5-f166-4bb4-866c-51c271f9c776 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.918576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f06e5-f166-4bb4-866c-51c271f9c776 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.918808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f06e5-f166-4bb4-866c-51c271f9c776 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.920361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 192/390] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:52.926253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:52.926538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:52.926809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:52.951417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:52.952675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:52.957673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.002894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac883e-535b-459f-b870-00063dcc8931 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.071269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.072083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.072410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.072703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.085219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:53.092432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 193/391] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:53.093287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:53.094142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:53.095539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:53.100469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 199/392] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:53.101609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.118152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.154042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.223959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73233b-93cd-461e-81bb-bd8b14df6bd6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.250456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6976e1-325a-4adf-99f7-e9741e0f36ff None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.262488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.263245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.263641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.263992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.264876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:53.265277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.267333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.277612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 194/393] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:53.285845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.317775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.319093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.319616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.320105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.323483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.334261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-38cf21fe-3758-44c6-839b-23ce82373875 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:53.342448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:53.343397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:53.344220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:53.347256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 200/394] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:53.349141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4104af-7bf4-4fe9-9475-58491b2db63a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.349604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4104af-7bf4-4fe9-9475-58491b2db63a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.350984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4104af-7bf4-4fe9-9475-58491b2db63a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.352516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 201/395] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:53.355760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4104af-7bf4-4fe9-9475-58491b2db63a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.428950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f018e-99c4-48a7-addc-bcec7bc60915 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.432889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.473641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.474939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.475289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.475646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.477339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.478876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee84a5e-4194-4a39-aaf3-145204ebef56 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.484389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 195/396] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:53.487022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.487316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.487596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.499670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4104af-7bf4-4fe9-9475-58491b2db63a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.505517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:53.506892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:53.538309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.539391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.539890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.540431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.541457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:53.542002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.543649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.553161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 202/397] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:53.554321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.592339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 196/398] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:53.592339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.644449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.690972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b61c12-7f96-4c69-aa4c-b058e5c5aaf4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.706053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbcbc0-fcc2-4f8f-9864-ab114a8fe51d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.716946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.734945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.736812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.737309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.737808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.739474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.741226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.743230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 203/399] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:53.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.746336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.747591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.748033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.749421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.751507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.753173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.760134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 197/400] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:53.760134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:53.884752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb0b7b-e0c2-494c-b2e6-3b9da4422f77 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.890711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8c146-4f61-4f5d-b99e-8e2cb1f2f953 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:53.922103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.923206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.923640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.923980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92/groups/b3f3503d679c4b70967b566b18280044/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.926076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] RBAC: Authorizing `identity:create_grant(project_id=37c60ab16e7e4d378e715a01c3873f92, group_id=b3f3503d679c4b70967b566b18280044, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:53.927028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:53.927947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:53.937123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:53.938197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:53.939195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:53.939747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:53.950644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:53.960073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:53.961569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:53.962386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:53.969223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 204/401] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:53.969223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.008412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 198/402] 10.222.0.47 () {58 vars in 1442 bytes} [Tue Jul 12 18:05:53 2022] PUT /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92/groups/b3f3503d679c4b70967b566b18280044/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:54.008412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.045111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.098566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce7d92-d3e8-4dd2-bb30-de42cd981118 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.136950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65096a-8307-47d4-9fd5-af130643789a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.138673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.139758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.140255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.140730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.142976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.144692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410eb76e-45c5-4ab3-90f0-9593e8364210 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.147080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 205/403] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:53 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:54.148656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63050a-e88b-4f56-82a2-81e8f99d58e2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.149124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63050a-e88b-4f56-82a2-81e8f99d58e2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.149577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63050a-e88b-4f56-82a2-81e8f99d58e2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.151026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 206/404] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:54.156619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63050a-e88b-4f56-82a2-81e8f99d58e2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.179954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.180674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.181070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.181355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.182125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:54.182821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.183841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df16881-459f-4c06-9dc5-b1f1e7ac9d51 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.194515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 199/405] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:54.195786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.196080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.197841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.215987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:54.217487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:54.225304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.287492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63050a-e88b-4f56-82a2-81e8f99d58e2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.301157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 200/406] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:54.305273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.343606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.343882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.345375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.345725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.346086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/groups/b3f3503d679c4b70967b566b18280044/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.348722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, group_id=b3f3503d679c4b70967b566b18280044, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.350371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.425268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b750a-6813-4ece-955f-216ef34695c7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.431716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 207/407] 10.222.0.47 () {58 vars in 1442 bytes} [Tue Jul 12 18:05:54 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/groups/b3f3503d679c4b70967b566b18280044/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:54.432654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.472880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.474424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.474948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.475408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.476977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.478744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.480942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 201/408] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:54.487288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.617952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfa4f4-00df-447d-af31-c72331f3ef33 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.642170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.655953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.656806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.657656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.657937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.672748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:54.682541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:54.683470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:54.684293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 208/409] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:54.691674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.693708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.706105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9f075-38a7-4d5b-90e0-46be679fc74d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.757070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.758013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.758301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.758580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.760079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.761700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.768765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 202/410] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:54.770582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.842275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e005e4-d697-43d7-bf02-26cce7c77a1e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.881915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.882673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.888329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.888329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:54.888329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:54.888329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:54.888329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:54.895667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 209/411] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:54.904441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:54.934376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a5542-b08c-416a-96cf-0daf04131fd9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:54.948361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:54.995255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:54.996452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:54.996876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:54.997364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/groups/b3f3503d679c4b70967b566b18280044/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.000646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] RBAC: Authorizing `identity:create_grant(project_id=2d5de8f3ff4c48169b2ef3f204ec2855, group_id=b3f3503d679c4b70967b566b18280044, role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:55.002862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd9d94-4df8-4515-870b-96ac0e8792bb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:55.025885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f552743-91e0-495e-8e26-bfcbad05dac0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.062784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.064765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.065253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.065690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] PATH_INFO: `/v3/groups/decaf77e9dfd461f9162a3c08c69f72d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.067165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] RBAC: Authorizing `identity:get_group(group_id=decaf77e9dfd461f9162a3c08c69f72d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:55.069032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:55.071180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 210/412] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:54 2022] GET /identity/v3/groups/decaf77e9dfd461f9162a3c08c69f72d => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:55.076766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:55.094582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 203/413] 10.222.0.47 () {58 vars in 1442 bytes} [Tue Jul 12 18:05:54 2022] PUT /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855/groups/b3f3503d679c4b70967b566b18280044/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:55.106893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23061fbe-8d88-4494-8d14-dd4cd6716f5a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.107179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23061fbe-8d88-4494-8d14-dd4cd6716f5a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.107467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23061fbe-8d88-4494-8d14-dd4cd6716f5a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.109120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 204/414] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:55.115219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.119855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.119855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.177532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:55.178765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:55.207569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:55.212371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2df8-1f30-4280-bf6c-aa06d727d7ae None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.215243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:55.262563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.263446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.263730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.264010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] PATH_INFO: `/v3/projects/cecdefe3d592442ba46417c3dd5f54f9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.265341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] RBAC: Authorizing `identity:get_project(project_id=cecdefe3d592442ba46417c3dd5f54f9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:55.267519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:55.269727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 211/415] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:55 2022] GET /identity/v3/projects/cecdefe3d592442ba46417c3dd5f54f9 => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:55.281505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:55.307682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 205/416] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:55.341209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:55.445864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a78b71-6b78-4364-9142-2bbcd6197f48 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.488915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.489900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.490257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.490602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] PATH_INFO: `/v3/projects/cecdefe3d592442ba46417c3dd5f54f9/groups/decaf77e9dfd461f9162a3c08c69f72d/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.492704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] RBAC: Authorizing `identity:create_grant(project_id=cecdefe3d592442ba46417c3dd5f54f9, group_id=decaf77e9dfd461f9162a3c08c69f72d, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:55.494991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:55.548355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4818-1683-4b7c-b1ff-dabd23cccb07 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.586768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.587681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.587976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.588282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.599425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:55.607786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:55.608768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:55.609712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 206/417] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:55.615961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:55.623798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 212/418] 10.222.0.47 () {58 vars in 1442 bytes} [Tue Jul 12 18:05:55 2022] PUT /identity/v3/projects/cecdefe3d592442ba46417c3dd5f54f9/groups/decaf77e9dfd461f9162a3c08c69f72d/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:05:55.636409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:55.757788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf070131-5fbb-4df9-aa3e-9ef848867d09 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.798018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.798956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.799245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.799526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:55.800300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:55.800626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:55.802048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:55.812945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 207/419] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:55.820063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:55.863149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:55.946887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764be5b-321d-4f34-9c4f-c1693a055900 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:55.990995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:55.991742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:55.992028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:55.992320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.089181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:56.090576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7567c-7df2-4743-8360-5de7a1b69524 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:56.133980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:56.221895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 213/420] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:56.227583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:56.339334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:56.380196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373b42f-bd5d-4ee6-aad2-5a366b2b11df None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:56.423087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42a702-e1e8-4a93-bdf2-58f56171f34b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.423409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42a702-e1e8-4a93-bdf2-58f56171f34b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.423713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42a702-e1e8-4a93-bdf2-58f56171f34b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.425199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 214/421] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:05:56.430201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:56.430338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.430610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.430897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.431810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.432358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.432922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.437050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:56.438628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:56.442177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 208/422] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:56 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:56.449769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:56.464461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:56.465672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:56.568823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc31257-be4c-4c20-a377-e84b068c5502 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:56.605178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:56.606102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.606393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.606675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.607917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:56.609641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:56.612637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 209/423] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:56 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:56.619350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:56.635498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:56.653882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 215/424] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:56.692684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:56.794155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc481f1-9320-437d-9e56-7d35650cc60c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:56.800658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:56.862119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:56.862708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.862997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.863280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.864130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=37c60ab16e7e4d378e715a01c3873f92) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:56.864468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:56.865916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:56.873237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9daa6d-d389-4a7e-9f1a-21576b4ac16a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:56.878701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 210/425] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:56 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 611 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:56.919688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:56.920533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:56.920828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:56.921104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:56.942538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2588734d-e2fc-412a-94a0-1ab248290b67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:56.950586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:56.951586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:56.953063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 216/426] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:56.958067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.051340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098016c3-d786-48e8-b1a8-79d9988d79c3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.051975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098016c3-d786-48e8-b1a8-79d9988d79c3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.052535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098016c3-d786-48e8-b1a8-79d9988d79c3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.054110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 217/427] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:57.059401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.060024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.060380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.060674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.087679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:57.089369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:57.102870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01d7fb-ce25-4db4-85b2-9b96452c25d7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.147125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.148224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.149322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.149817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.150960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] RBAC: Adding query filter params (name=member) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:57.151481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.153191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.163689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 211/428] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:57.168905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.178798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 218/429] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:05:57.204864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.269422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.303277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c78e-ece0-4088-a409-96a86038e224 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.336500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c52d16-6137-4252-8aca-a37511b99415 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.340262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.346480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.351433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.351433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.351433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.379020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.380226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.380777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.381234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.396335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:57.403423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:57.404324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:57.405143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 219/430] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:05:57.411962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.425974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.427728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.522124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0833b33-a337-442c-becc-06bc7ad110e1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.531741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 212/431] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:57.536536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.538297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.559429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.560106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.560472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.560704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.561767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:57.561874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.563319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.576936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 220/432] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:57.577973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.584669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.669659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93193bba-d7c0-4469-a3aa-73ae8e60a940 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.697304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bfbd2-57f0-43b7-9d85-bc2b44ff6a2b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.716763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.717629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.717915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.718199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] PATH_INFO: `/v3/projects/37c60ab16e7e4d378e715a01c3873f92` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.719444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] RBAC: Authorizing `identity:get_project(project_id=37c60ab16e7e4d378e715a01c3873f92)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.720936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c7386-fb53-43c4-bdfb-971d362bb191 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.724353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 213/433] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/projects/37c60ab16e7e4d378e715a01c3873f92 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:57.724353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67288b47-96bd-4a3d-a8cb-03e20ede0c0e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.724353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67288b47-96bd-4a3d-a8cb-03e20ede0c0e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.724626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67288b47-96bd-4a3d-a8cb-03e20ede0c0e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.725964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 214/434] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:57.730445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67288b47-96bd-4a3d-a8cb-03e20ede0c0e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.738775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.739722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.740232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.740836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.789126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.814658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.816131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.833335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:57.836791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67288b47-96bd-4a3d-a8cb-03e20ede0c0e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:57.875169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:57.876968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.876968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.877234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.878439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:57.879898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222980d-6268-41a2-b4cc-52f3a65578ab None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:57.881568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 215/435] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:57.882783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:57.883076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:57.883356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:57.900329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:57.901552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:57.933031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 221/436] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:57.934146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:57.979576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 216/437] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:57.980525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.025014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.053435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7a02f-6595-4472-9b81-c5bc4be0074e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.089621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.090284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.090554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.090900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.091751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=37c60ab16e7e4d378e715a01c3873f92) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:58.092120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.093630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.101069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 222/438] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=37c60ab16e7e4d378e715a01c3873f92&role.id=5a2774adec8f4206a81f758406367b9f => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.101971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.113019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.118293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa3079-346e-4ef5-b3d4-5adf77bd0c75 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.159443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.160371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.160661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.160938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.162435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.163944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e19d1-9ab0-4203-9321-d0b7e939f0d2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.165960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 217/439] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:57 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.167227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78ee8-cfdf-4792-a35a-a429b5a19606 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.167516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78ee8-cfdf-4792-a35a-a429b5a19606 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.167795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f78ee8-cfdf-4792-a35a-a429b5a19606 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.173075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 218/440] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:05:58.173075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f78ee8-cfdf-4792-a35a-a429b5a19606 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.225339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17227-0b9f-49d2-985a-8d3eb519c976 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.268198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.269133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.269499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.269784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.278104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.282098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:58.289598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f78ee8-cfdf-4792-a35a-a429b5a19606 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c96bfdd8-961b-490a-b3c6-c5f4df2f0d85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:58.290725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:58.291588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:05:58.292450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 223/441] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:58.294262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.294562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.294848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.314043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:05:58.315245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:05:58.343481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.345615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.346227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.346509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.347726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.349474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.351108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 219/442] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.351861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.393581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.397645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 224/443] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:05:58.398508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.471613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58fb62a-c06f-4666-89b9-7e4a03e28517 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.512538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.515595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.515595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.515595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.515595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:58.515595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.516109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.526511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 220/444] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.527357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.542309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172579d-b5cd-4037-b353-5bfec5dac689 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.577322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.588747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.590425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.590425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.590425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.590907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=5a2774adec8f4206a81f758406367b9f, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:58.591228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.592895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.601796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 225/445] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=5a2774adec8f4206a81f758406367b9f => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.602894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.651272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.657790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6c23d-ab1b-408f-9dcc-a0afbe12447c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.697774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.698582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.698868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.699150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.716839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:05:58.728525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:05:58.729514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:05:58.730307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:05:58.736466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 221/446] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:05:58.738637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.740681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6504c14-ba8b-4d59-86fc-28e886d69b54 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.777171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.777748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.778039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.778320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.848596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.860227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.862010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.862904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a185a-3e88-4287-9fd9-bd890fbe61d1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:58.903927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:58.904546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:58.904836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:58.905115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:58.905939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:58.906249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:58.908262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:58.920357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 222/447] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:05:58.922713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:58.946216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:58.985676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 226/448] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:05:58.991807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.027723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a66350-1b49-4bf4-ad98-9909726c31be None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.060511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.061160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.061445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.061723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.084838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:59.093668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fea730-57b9-4cc5-a76f-228d0c6eaae6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.131009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.131871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.132156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.132465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] PATH_INFO: `/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.133556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.133667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] RBAC: Authorizing `identity:get_project(project_id=2d5de8f3ff4c48169b2ef3f204ec2855)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.135018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.135107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.136766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 227/449] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/projects/2d5de8f3ff4c48169b2ef3f204ec2855 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:59.142802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.153271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:59.234969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 223/450] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 18:05:59.240518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.241026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8bbdb-52ef-4b52-9804-99a28789fb79 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.275899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.276778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.277070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.277349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] PATH_INFO: `/v3/groups/b3f3503d679c4b70967b566b18280044` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.278594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] RBAC: Authorizing `identity:get_group(group_id=b3f3503d679c4b70967b566b18280044)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.279982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.281996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 228/451] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:59 2022] GET /identity/v3/groups/b3f3503d679c4b70967b566b18280044 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:59.287530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.314341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:59.337252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97936b2d-d299-4f3b-baa1-273b51add352 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.354190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:59.371426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.372296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.372843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.373128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] PATH_INFO: `/v3/projects/cecdefe3d592442ba46417c3dd5f54f9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.374343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] RBAC: Authorizing `identity:get_project(project_id=cecdefe3d592442ba46417c3dd5f54f9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.376672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.378378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 224/452] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:05:59 2022] GET /identity/v3/projects/cecdefe3d592442ba46417c3dd5f54f9 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:59.383705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.390982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c4617-47ef-4cdb-bf68-6e3c387343a9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.427024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.427692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.427986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.428271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.429174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] RBAC: Adding query filter params (group.id=b3f3503d679c4b70967b566b18280044, role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:59.429502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.430981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.436897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 229/453] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:59 2022] GET /identity/v3/role_assignments?group.id=b3f3503d679c4b70967b566b18280044&scope.project.id=2d5de8f3ff4c48169b2ef3f204ec2855&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:59.489309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f918-c4ee-45b7-a873-dcf19dcfa372 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.525816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.526697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.526986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.527265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] PATH_INFO: `/v3/groups/decaf77e9dfd461f9162a3c08c69f72d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.528530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] RBAC: Authorizing `identity:get_group(group_id=decaf77e9dfd461f9162a3c08c69f72d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.530081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744906bf-1da3-4c83-8a1d-7eb866af5b47 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.531687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 225/454] 10.222.0.47 () {56 vars in 1180 bytes} [Tue Jul 12 18:05:59 2022] GET /identity/v3/groups/decaf77e9dfd461f9162a3c08c69f72d => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:05:59.537975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:05:59.638116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:05:59.647227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f275ea-072c-4fa0-a1a2-e9a7a1d78780 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:05:59.685089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:05:59.686050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:05:59.686537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:05:59.686827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:05:59.687717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] RBAC: Adding query filter params (group.id=decaf77e9dfd461f9162a3c08c69f72d, role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=cecdefe3d592442ba46417c3dd5f54f9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:05:59.688166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:05:59.690347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac75d7e-d50f-4d09-b6ef-e70b33924c95 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:05:59.698642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 230/455] 10.222.0.47 () {56 vars in 1376 bytes} [Tue Jul 12 18:05:59 2022] GET /identity/v3/role_assignments?group.id=decaf77e9dfd461f9162a3c08c69f72d&scope.project.id=cecdefe3d592442ba46417c3dd5f54f9&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:02.345731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973420-ba86-40fc-99d7-802cf9c08fe7 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.346595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973420-ba86-40fc-99d7-802cf9c08fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.347028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e973420-ba86-40fc-99d7-802cf9c08fe7 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.349074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 226/456] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:02.355007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.355781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.356236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.377051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:02.378510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:02.481809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d173b5-a210-42ec-9ff2-ed8f011f8c02 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.482115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d173b5-a210-42ec-9ff2-ed8f011f8c02 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.482374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d173b5-a210-42ec-9ff2-ed8f011f8c02 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.486508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 227/457] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:02.495184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 231/458] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:02.513867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74f364-fb56-4b8a-8c05-57b2d1073423 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.514158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74f364-fb56-4b8a-8c05-57b2d1073423 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.514437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74f364-fb56-4b8a-8c05-57b2d1073423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.531571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:02.543971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3d74f364-fb56-4b8a-8c05-57b2d1073423 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:02.549474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3d74f364-fb56-4b8a-8c05-57b2d1073423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:02.636287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:02.651596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:02.655252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f04c8c-05d9-42ae-971f-05b143a0b78a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:02.675334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 228/459] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:02.704437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:02.705259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.705709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.706005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.724347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:02.725965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4118-7074-4769-a6da-96cd5b6335a2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.726524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4118-7074-4769-a6da-96cd5b6335a2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.726923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4118-7074-4769-a6da-96cd5b6335a2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.729511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:02.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 229/460] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:02.732114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4118-7074-4769-a6da-96cd5b6335a2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-acb0bdd5-dba8-41d6-98dc-0b6ff8967742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:02.751479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:02.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:02.753609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:02.753609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:02.753609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:02.753609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:02.753609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 232/461] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:02.754376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.754731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.754857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.778421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:02.779623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:02.840897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4118-7074-4769-a6da-96cd5b6335a2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:02.858088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 233/462] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:02.862487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:02.905865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:02.907152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:02.907534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:02.907763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:02.921944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:02.923917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:02.949789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:02.951376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:02.952277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:02.953460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:02.953460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:02.953460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:02.953460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:02.953460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 230/463] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:02.959082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:02.996954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797a59c-75e4-4dc0-9f71-2866becf12be None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:02.998870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.034971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.036093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.036642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.037127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.038079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:03.038591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.040168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.053863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 234/464] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:03.055026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.071951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa326-df97-46d0-882f-7134f5d329ce None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.111114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.111914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.112298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.112610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.123849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.125704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:03.149529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:03.150426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:03.151284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:03.151284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:03.151284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:03.151284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:03.152044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 231/465] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:03.152912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.167094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e5bb-f588-445e-8dca-a2312c6bd100 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.195915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.204965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.205730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.206313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.206313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.207015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:03.207328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.208772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2aac54-0b54-43b4-9a03-61b9737e0f0f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.221031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 235/466] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:03.222331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238af09-69cf-414d-8920-acad363b1001 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.222629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238af09-69cf-414d-8920-acad363b1001 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.222910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238af09-69cf-414d-8920-acad363b1001 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.224635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 236/467] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:03.229077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238af09-69cf-414d-8920-acad363b1001 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.254578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.267621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df180257-07d1-4eef-acb5-d2ec09840c36 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.303655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.304355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.304627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.304907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.305689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.307143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8251-2624-4332-bc77-fe0e25513eb8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.343301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238af09-69cf-414d-8920-acad363b1001 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.379424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.380165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.380397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.380684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.381416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:03.381727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.383103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.400039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 237/468] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:03.400039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.408815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 232/469] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:03 2022] POST /identity/v3/users => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:03.410820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.411313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.411805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.430875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:03.432449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:03.465534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.520523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3cda6-e405-4c12-a879-1958ae96df22 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.536742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 233/470] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:03.538068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.561609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.562801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.563559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.564079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.565075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.566898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.639725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.654607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 238/471] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:03 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:03.655271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.681684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6765f910-8bc2-4140-9107-01402f7931bb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.731736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.732601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.734441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.734441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.734441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.735617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.787805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4631185-426a-48c7-847a-7a0bf2dca3a7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:03.790612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:03.827758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:03.829221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.829673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.829976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.841833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:03.842898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-41216d6e-6a10-4b07-841f-cf5bb56615f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:03.863441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:03.864401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:03.865232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:03.865232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:03.865232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:03.865232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:03.867206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 239/472] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:03.868827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450521-3d41-485c-a12d-91d288c3411b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.869322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450521-3d41-485c-a12d-91d288c3411b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.869746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 234/473] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:03 2022] POST /identity/v3/users => generated 248 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:03.870017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450521-3d41-485c-a12d-91d288c3411b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.871730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 240/474] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:03.873390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:03.878332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:03.878632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:03.878901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:03.907436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:03.910071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:04.011991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283efe7a-ec80-45dd-b7a9-bcafc0691a19 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:04.013911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 241/475] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:06:04.048436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:04.090654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:04.102783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:04.103468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:04.103755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:04.108221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:04.108221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:04.108221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:04.108221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:04.121526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 235/476] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:04.128708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:04.184849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:04.202891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b23009-2973-4da8-9278-060b7b5a7d4d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:04.249720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:04.250632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:04.251204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:04.251655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:04.252607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:04.254153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:04.274486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e6dbc-d5c5-4a29-a632-ea9a715b08fd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:04.291738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.sql.core [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 18:06:04.291738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 18:06:04.291738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [parameters: {'id': '770a7d2ceef54cabb7b8bcaaffdc7510', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 18:06:04.291738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80576) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 18:06:04.312628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8834-a736-4015-a97e-897209816027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:04.313632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8834-a736-4015-a97e-897209816027 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:04.314067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8834-a736-4015-a97e-897209816027 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:04.314550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a8834-a736-4015-a97e-897209816027 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:04.315410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a8834-a736-4015-a97e-897209816027 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:04.316895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:04.317923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application yield resource Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise self.value Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application session.commit() Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 18:06:04.319076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self.session.flush() Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application compat.raise_( Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 18:06:04.320121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:04.321207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:04.322227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [parameters: {'id': '770a7d2ceef54cabb7b8bcaaffdc7510', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:04.323198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 18:06:04.324173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.328140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 242/477] 10.222.0.47 () {60 vars in 1131 bytes} [Tue Jul 12 18:06:04 2022] POST /identity/v3/roles => generated 144 bytes in 279 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:04.330617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34a8834-a736-4015-a97e-897209816027 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:04.331899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:04.429653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 236/478] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:04 2022] POST /identity/v3/users => generated 249 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:04.457094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90ae35-c472-4c40-ab60-ba26d8f939ce None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:04.497554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:04.498439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:04.498780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:04.499112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:04.511384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-59ee43ba-0e6f-440f-90b5-ddec97e7e863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:04.520812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:04.521759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:04.522568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 243/479] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:04.528409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8834-a736-4015-a97e-897209816027 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:04.619296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:04.682343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a8834-a736-4015-a97e-897209816027 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:04.716698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:04.717379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:04.717660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:04.717935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:04.718692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:04.718998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:04.721737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d6e97-86b5-4525-907d-14d319472b39 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:04.731457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 237/480] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:06.095225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c375-2be0-4e11-adc8-da36bc137508 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.095777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c375-2be0-4e11-adc8-da36bc137508 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.095777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962c375-2be0-4e11-adc8-da36bc137508 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.097452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 244/481] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:06.102865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.103283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.103491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.129426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:06.130660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:06.167943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384340-390f-409e-bd88-904de0e140d5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.168269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384340-390f-409e-bd88-904de0e140d5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.168558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384340-390f-409e-bd88-904de0e140d5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.170148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 245/482] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:06.175637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.175927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.176216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.194320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:06.195507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:06.200820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:06.241495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 238/483] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:06.277887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.312518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 246/484] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:06.335279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:06.363908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.451660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730b915-ffa9-4cb6-b8df-082608071c66 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:06.515417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:06.516844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.517128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.517598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.530806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-fbe2f4f0-c97a-4238-8c3f-61d6c886c9dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:06.539229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:06.540143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:06.541217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 239/485] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:06.548901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3fa8-820b-44fe-8f36-4be4073adeaf None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.549271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3fa8-820b-44fe-8f36-4be4073adeaf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.549621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c3fa8-820b-44fe-8f36-4be4073adeaf None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.550520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8d788-fd59-40fc-9df2-e580636a92a9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:06.551146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 240/486] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:06.552663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c3fa8-820b-44fe-8f36-4be4073adeaf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.591492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:06.592299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.592613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.592895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.601621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:06.605136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3bb54104-58a4-4532-b67f-073e3369658a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:06.620867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:06.621786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 247/487] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:06.622572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.623276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.623276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.640609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:06.642352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:06.676633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:06.682896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c3fa8-820b-44fe-8f36-4be4073adeaf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:06.724333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 248/488] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:06.724333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.732487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:06.733646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.734142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.734711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.735692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:06.736232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:06.738140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:06.760174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 241/489] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:06.762363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.833127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3b2d0-b205-4871-8f6b-d5ff058d4dd3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:06.864450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:06.869611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:06.875547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:06.878824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87213-181c-428b-a776-2a16027c8003 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:06.882155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 249/490] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:06.882991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.918463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:06.919690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:06.920175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:06.920774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:06.933117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:06.941744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:06.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:06.949839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:06.950957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 242/491] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:06.952122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:06.978240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.003055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65727d75-6459-4935-a815-ff299a7c481e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.041583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.042375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.044454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.044454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.055236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.056735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.067184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4df980-b901-411d-8c5d-271953c2708d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:07.074243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.075122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.075948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:07.075948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:07.075948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.075948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:07.075948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 250/492] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.080605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.111341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.112082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.112413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.112730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.127171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.128980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.132892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4f9ea88f-97de-493d-9233-b32c46b19d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:07.149806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 243/493] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c0b4-bac9-41e4-883a-91f993d52b1c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c0b4-bac9-41e4-883a-91f993d52b1c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3c0b4-bac9-41e4-883a-91f993d52b1c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.156467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 244/494] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:07.157649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3c0b4-bac9-41e4-883a-91f993d52b1c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.201414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36b2c6-13a3-444c-a606-286f6727a2ca None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.231212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.236436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.237105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.237398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.237679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.238428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:07.238738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.240120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.248028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 251/495] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:07.248862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.267392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3c0b4-bac9-41e4-883a-91f993d52b1c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.301334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.301999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.302290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.302563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.303285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:07.303589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.305002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b8029-1914-4e86-803a-050721f6c9d3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.317321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 245/496] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:07.318640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f84e8-c7d3-4c2a-9fc2-fe3e86d6fb34 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.318941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f84e8-c7d3-4c2a-9fc2-fe3e86d6fb34 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.319231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f84e8-c7d3-4c2a-9fc2-fe3e86d6fb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.337050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0c4f84e8-c7d3-4c2a-9fc2-fe3e86d6fb34 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:07.338352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0c4f84e8-c7d3-4c2a-9fc2-fe3e86d6fb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:07.354416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d47c09-6cef-47f2-a3f7-b3c49c3fd845 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.361821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.389572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.390238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.390531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.392677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:07.392996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.394384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.420340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 246/497] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:07.420518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 252/498] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:07.420518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.420645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b6d27-0b41-4713-ba45-ede2a573bfa3 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.420645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b6d27-0b41-4713-ba45-ede2a573bfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.420645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b6d27-0b41-4713-ba45-ede2a573bfa3 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.420645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 247/499] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:07.420645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b6d27-0b41-4713-ba45-ede2a573bfa3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.476249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.517635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b6d27-0b41-4713-ba45-ede2a573bfa3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.524062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c007b6-a2a4-4fc5-8cc9-9e680a281371 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.550508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.551304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.551594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.551876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.557665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.558454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.558746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.559033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.559301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.560808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.570546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.572003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:07.576487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.577386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:07.578282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 248/500] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.579010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-813fcdc6-5591-4a0e-8996-593facbcd2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:07.593306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.594312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.595327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:07.595327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:07.595327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:07.595327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:07.595327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 253/501] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.597441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.597739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.598030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.614974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:07.616179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:07.630308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.680838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c2948-7605-4d2f-b4fb-159c2a3aac3e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.684525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 254/502] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:07.685175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.704111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.716031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.716872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.717166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.717450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.726436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.727905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:07.743448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.744484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:07.745396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 249/503] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.745964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.814093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8479f2-71c1-43e6-89ca-e7d580e305da None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.845256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d63fbc-5fe7-4acc-8964-d082c31961bb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:07.847438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.848525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.848814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.849094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.859191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:07.867722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:07.868786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:07.869583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:07.873690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 255/504] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:07.874505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.879033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:07.879701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:07.879985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:07.880261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:07.881145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:07.881451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:07.882898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:07.885958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.918387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 250/505] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:07.919216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:07.962459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:07.976477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8a30-18bc-443e-a01e-3dbd429567c2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.009945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.010610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.010901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.011181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.011905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:08.012216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.014147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.019455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37767-d38b-4bbc-ad6d-3640cea314a5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.026292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 256/506] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:08.027092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.059253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.060049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.060368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.060651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.070866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.072357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:08.090745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:08.091764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:08.092729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:08.092729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:08.092729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:08.092729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:08.092729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 251/507] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:08.093059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.095476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.130556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff2984-aa69-4077-98f7-f2f064988f29 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.165563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.166234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.166525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.166806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.167656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:08.167977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.169477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.183903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.192118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d15a00-4ae4-4b98-815f-2eba00050c58 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.210769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 257/508] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:08.210769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.232229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.232878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.233170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.233448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.234202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:08.234521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.236533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.244345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 252/509] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:08.245095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.308637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f67b2-a9b9-41df-8ed9-6c523239f62a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.334415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.343008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.343663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.344712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.344993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.347534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434376f-7031-408c-9b8d-dd09f4f34704 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.381769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.382565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.382850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.383131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.390712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.392184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:08.407341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:08.409348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 253/510] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:08.411554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.419805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.421329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.430627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.520519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d367195-5205-4f63-8c6d-b6e5aa806084 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.525251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 258/511] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:08.530313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.553031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.557597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.558258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.558543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.558821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.559554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:08.559861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.561281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.573446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 254/512] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:08.574241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.628160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e39c08-9775-493f-a4ec-24929e5a15cc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.648069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.662651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.663329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.663621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.663901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.673180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93956712-8bc8-408d-b9fb-eefa3e32300c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.706025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.706810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.707093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.707374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.718843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.721090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.732463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.733908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:08.738885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:08.739904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 255/513] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:08.745395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.767551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.838576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65fb6d-2f9a-43fc-bc33-d391bcfa4688 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.840021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 259/514] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:08.840923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.865415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.876822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.877494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.877779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.878060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.878909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:08.879218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:08.880684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:08.897999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 256/515] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:08.898808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:08.951279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a303e4-2692-485a-a512-a7bfcdbe0d35 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:08.976813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:08.986651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:08.993100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:08.993100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:08.993100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:08.999319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.000053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60e85f-2bc5-49fa-b071-ef984d8264a9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.001099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:09.018796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:09.019714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:09.020763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:09.020763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:09.020763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:09.020763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:09.020763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 260/516] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:09.021029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.037004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.037294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.037577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.038292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.039762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.119415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2e2db-691e-4daa-9504-c5ce9eb14853 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.127118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 257/517] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:08 2022] POST /identity/v3/users => generated 247 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:09.128039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.158084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.158737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.159054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.159330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.160054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:09.160429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.161883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.179498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 261/518] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:09.180543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.184040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.230807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.249919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be1da2-9c07-455c-8a7c-48f73585c80f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.290362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.291302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.291451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.291729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.304686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.305944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.306457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d147de1-b096-434e-80b1-81338e8e4115 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:09.324183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:09.325224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:09.326060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:09.326060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:09.326060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:09.326060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:09.326860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 258/519] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:09.328898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.350933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.351777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.352140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.352593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.432562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.443925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c953d35-e06e-44b9-b68d-033583bea142 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.447915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.450705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.489712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.490390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.490682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.490970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.491718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:09.492032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.493472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.510757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 259/520] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:09.510757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.513003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.581381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 262/521] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:09.581381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.654421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bbb6e-7647-4f5d-8c87-96c6a759a1f4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.691792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.692558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.692847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.693136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.700014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098bff5-34c3-43c0-a476-7c904b7c8c8d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.701436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.702958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:09.719630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:09.721151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 260/522] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:09.723798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.740587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.741280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.741572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.741860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.742597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:09.742908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.743014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.744433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.753156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:09.764666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 263/523] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:09.765620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.862799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480b34d-3bc2-4791-9cbf-a9db09a55a6e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.915489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.916752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.917220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.917703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.921848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae295f5-2140-4322-9194-af308d2bc3d8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:09.929183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.931710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:09.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:09.957374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:09.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 261/524] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:09.959306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:09.981323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:09.982124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:09.982439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:09.982723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:09.997182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:09.999642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:10.023965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:10.025669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 264/525] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:10.027450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.059591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.086394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.096158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43bbc-a436-470b-a699-8b1c135f6954 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.133223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.134166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.134459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.134748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.135613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:10.135924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.137424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.144384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9f41d-aca1-4ef6-823f-44282d159fef None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.158968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 262/526] 10.222.0.47 () {56 vars in 1139 bytes} [Tue Jul 12 18:06:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:10.159822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.188042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.189468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.189940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.190373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.197926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.199566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:10.215906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:10.216897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 265/527] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:10.217852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.283924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f90350-ca00-4db3-9556-b3ec4ffc3241 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.321704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.330269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.345494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.345494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.345494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.345494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:10.345494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.346158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa3b19-4e26-4eb2-85db-123224cfff50 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.346287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.348878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.364356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 263/528] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:10.364356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.394640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.397127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.397127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.397127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.398490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:10.399003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.402155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.428657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 266/529] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:10.430530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.512954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07881c0-fc76-4101-9de0-ab25b1ab0fc7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.565675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.565865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765270c-c3c6-422a-b483-be105d4bd828 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.566681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.567146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.567607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.619005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.620041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.620646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.621123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.622290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:10.622774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.625389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.654939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 267/530] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:10.656007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.665418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.683894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.686630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.700271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.792752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3de1c6-46aa-4236-8111-38bf94ee0c5a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:10.816492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 264/531] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:10.817922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.836096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:10.836858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:10.837198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:10.837597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:10.838511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:10.838868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:10.840777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:10.848072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 268/532] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:10.849808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:10.944682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:10.952823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd122b5-5f13-419f-9ef4-6b85b64b4cc6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.014930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.016249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.016768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.017244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.030278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.033439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:11.058114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:11.060023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:11.062214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:11.064397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 265/533] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:11.065975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.068357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:11.077693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b532f7-de29-4356-8d75-fd559b8c75da None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.122223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.123021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.123373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.123742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.125438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=b32ef598390145d1b88c3e3811edb79b) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:11.125612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.127143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.134819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 269/534] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:10 2022] GET /identity/v3/role_assignments?user.id=b32ef598390145d1b88c3e3811edb79b&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:11.135942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.262059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab9a6e-bc87-4303-9ad9-732a0dd8d960 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.345890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.346820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.347299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.347726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.349306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:11.350316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.352719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d813c2cd-b673-4fd0-8640-f1cd36292c82 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.378304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafbb13-b381-46b4-b7b9-8698b7c7e60d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.383473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 270/535] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:11.387339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37e8d-88a2-4895-97ef-e4240f18fbd8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.389533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37e8d-88a2-4895-97ef-e4240f18fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.389533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37e8d-88a2-4895-97ef-e4240f18fbd8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.393660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 271/536] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:11.395636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb37e8d-88a2-4895-97ef-e4240f18fbd8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.428732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:11.432225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.433722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.434009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.434288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.452850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.455515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bfae7a3d-96f5-43f8-9348-72aea205dbee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:11.478855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:11.479781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:11.480647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:11.480647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:11.480647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:11.480647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:11.483373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 266/537] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:11.485724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.486396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.486871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.494459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:11.510162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:11.511956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:11.569946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb37e8d-88a2-4895-97ef-e4240f18fbd8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.601670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 267/538] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:11.602832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.641754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.643111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.643862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.644320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.646699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=44fe9acbbbdc463ab22869886df4e3aa) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:11.647202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.649585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.665981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 272/539] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/role_assignments?user.id=44fe9acbbbdc463ab22869886df4e3aa&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:11.667113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.804905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:11.821485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f5f34-476e-49a3-bbb0-e6bfbc51dc92 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.834257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475630bb-bc03-45c6-bae7-15f08685460b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:11.922229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:11.929491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.930359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.930618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.939521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.939521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:11.939521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:11.939877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:11.939877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:11.940116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:11.940219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:11.940219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:11.955016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 268/540] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:11.962277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:11.973649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:11.981724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:11.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:11.984391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:11.988811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 273/541] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:12.000873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:12.145034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7630bc-9fd6-45c7-9141-2c2706cd94c1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.190116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578b33b-bce5-4010-98c3-4aff4c93a612 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.205948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.206762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.207050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.207695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.215466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.216966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:12.233184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:12.235071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:12.236930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 269/542] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:12.245669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:12.275621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.276545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.276757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.276957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.279378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:12.279711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.281181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.299561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 274/543] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:12.307722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:12.316177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:12.343684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:12.419514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92708b01-b556-4ede-8ef8-372601b7818b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.465510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c629a-cca3-4df8-b767-5747a871690c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.492864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.493830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.494285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.494711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.495663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:12.496672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.499325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.518287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 270/544] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:12.523249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.524051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.525191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.525191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.527795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:12.547978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.549786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:12.580842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:12.581750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:12.582573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:12.582573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:12.582573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:12.582573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:12.590046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 275/545] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:12.597200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:12.695804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee186a-f369-476a-9e44-3795468a7240 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.723444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:12.727925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:12.774056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.775036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.775468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.775905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.779777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=bd67da2713214172989b5c689a8885f4) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:12.780290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.783282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.796697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 271/546] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:12 2022] GET /identity/v3/role_assignments?user.id=bd67da2713214172989b5c689a8885f4&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:12.812501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677db72-12f4-4984-bd1a-5a4bab55d879 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:12.883990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:12.884719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:12.885010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:12.885316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:12.886075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:12.886598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:12.887815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:12.900908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 276/547] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:12.906010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:13.068500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66f2e-7919-4cba-a4f9-74824e257756 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:13.132080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:13.133336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:13.133767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:13.134194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:13.158128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:13.160644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:13.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:13.193117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:13.194013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:13.198321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 272/548] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:13.203465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:13.336213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc4930-b16d-438b-9739-c33f8cf5865b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:13.357054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:13.371409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:13.372099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:13.379734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:13.379734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:13.379734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:13.379734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:13.379734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022235d2-c9ce-471e-a71b-0d4bfd2fae4e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:13.394656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 277/549] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:13.400164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:13.429865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb3f1-8e2b-4de9-b0cd-f94b28d528e4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:13.430307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb3f1-8e2b-4de9-b0cd-f94b28d528e4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:13.430735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb3f1-8e2b-4de9-b0cd-f94b28d528e4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:13.434687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 278/550] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:13.439959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:13.440425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:13.440854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:13.468673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:13.484080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:13.490887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:13.562469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6e818-b31b-431f-b85e-d863d693bbc2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:13.626262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:13.626924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:13.627211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:13.627489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:13.635453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 279/551] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:06:13.674244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:13.827829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:13.831361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:13.937383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:13.941863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b76bdd-6f6e-47a8-acd3-037ab0982bf9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:13.960643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.033681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.034501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.034795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.035080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.062172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:14.087587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:14.088591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:14.089384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:14.090263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 280/552] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:14.097563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.117058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 273/553] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:14.125903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.261135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424dccf-ef55-47c8-85e3-8d9027484330 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.267505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fac021-1d83-4055-a871-4e3396d86041 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.306131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.307137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.307731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.308269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.309393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:14.309870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.310730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.311668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.311915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734659-e722-4a79-92f8-1bac23e5e01c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.312028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.312232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.316462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.323040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 281/554] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:14.324661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7709ac4-9ce4-479d-b37c-e88c5db61ad0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.333540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7709ac4-9ce4-479d-b37c-e88c5db61ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.333540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7709ac4-9ce4-479d-b37c-e88c5db61ad0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.333738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.333827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 282/555] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:14.333934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.334073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7709ac4-9ce4-479d-b37c-e88c5db61ad0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-25520971-a7af-4b4c-947e-419077cd1eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:14.353782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:14.354907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:14.356090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:14.356090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:14.356090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:14.356090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:14.361857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 274/556] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:14.361857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.361857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.361857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.374498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.380284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:14.381746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:14.461920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7709ac4-9ce4-479d-b37c-e88c5db61ad0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.476543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 275/557] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:14.476543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.498019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.498816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.499101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.499381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.510244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.511757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3e766c32-70f3-4424-860a-3c089dfa37dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:14.529794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:14.531196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:14.532120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:14.532120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:14.532120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:14.532120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:14.532120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 283/558] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:14.532892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b539860-2b6d-46a5-918e-1525de9e569c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.533186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b539860-2b6d-46a5-918e-1525de9e569c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.533465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b539860-2b6d-46a5-918e-1525de9e569c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.538735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 284/559] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:14.538735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b539860-2b6d-46a5-918e-1525de9e569c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.595007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054f5be-4fa1-4cd8-b729-7e27cd5c50aa None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.614129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.621456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.632393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31597588-8c95-4282-af04-90685643150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.633069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597588-8c95-4282-af04-90685643150a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.633368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597588-8c95-4282-af04-90685643150a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.637035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597588-8c95-4282-af04-90685643150a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.637035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597588-8c95-4282-af04-90685643150a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:14.637035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597588-8c95-4282-af04-90685643150a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.637035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597588-8c95-4282-af04-90685643150a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.643619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b539860-2b6d-46a5-918e-1525de9e569c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.652662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 276/560] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:14.652662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31597588-8c95-4282-af04-90685643150a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.683069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.684033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.684416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.684792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.697528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-df5a2bb9-047b-4ba2-9eb3-ac3f87fb50b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:14.706081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:14.707089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:14.707931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 285/561] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:14.708652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.709588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.709693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.727673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:14.728892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:14.766766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31597588-8c95-4282-af04-90685643150a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.802307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 286/562] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:14.804744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.805424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.805717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.806002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.806745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:14.807061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.808476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.813265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.826488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 277/563] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:14.827418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.870820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.895756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:14.933427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005ef10-9904-4372-a8e9-adb15c1dcd4e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.934883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677bc20-5842-42fc-a17b-4d1be08288bb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:14.967492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.967880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:14.968685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.969382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:14.969485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.969748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.970056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:14.970480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:14.971372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:14.971824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.973747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.976875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:14.978315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:14.987132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 278/564] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:14.989192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:14.994702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:14.995578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:14.997073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 287/565] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:14.997574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.092243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.096808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fbc4b-9e17-4884-8d27-d1ce7e28d53f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.106647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48949-cbbf-432c-a03e-f577f08c3b0a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.134345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.135430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.135718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.135988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.139416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.144268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.145932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.147373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.148179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.148507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.148794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.160099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:15.162321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:15.163265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:15.164205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 279/566] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:15.165421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:15.173802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:15.174756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:15.175591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 288/567] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:15.177030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.273057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb30154-249a-4a3b-b509-699134984f30 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.283294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dff65-35e5-47ef-8f12-6600894ece95 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.305909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.306697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.306984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.307261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.316801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.317457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.317746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.318026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.318426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.318847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:15.319157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.319885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.320611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.332516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 289/568] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:15.333325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:15.337145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:15.338193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:15.339271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:15.339271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:15.339271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.339271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:15.339271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 280/569] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:15.339615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.355943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.384770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.437342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dba62-e027-4865-b714-fe139dd8327e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.440856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff750361-89f4-49aa-a6ce-9e2e3e61cee2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.477227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.477520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.477967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.478174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.478275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.478453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.478544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.478728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.479304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:15.479517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:15.479611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.480340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.481084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.481717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddd333-7d51-4de7-a20b-b54c8fd5d5db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.491453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 281/570] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:15.492811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5779-de9d-4920-8dd9-bfb3cf4b3e7f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.493114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5779-de9d-4920-8dd9-bfb3cf4b3e7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.493393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c5779-de9d-4920-8dd9-bfb3cf4b3e7f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.494766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 282/571] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:15.495349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c5779-de9d-4920-8dd9-bfb3cf4b3e7f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 290/572] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:15.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.571571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.596440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c5779-de9d-4920-8dd9-bfb3cf4b3e7f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.601230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e568f09-ce4f-4ad7-b0a3-7cebbe19e1b4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.608847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.630730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.631384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.631671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.631953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.632824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=fe3f01883b804652b396cd4e28ab4b46) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:15.633145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.634600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.634984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.635643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.635928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.636209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.636957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:15.637269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.638672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa9f50-7dea-4c2f-99af-778bf733f104 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.640297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 283/573] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/role_assignments?user.id=fe3f01883b804652b396cd4e28ab4b46&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:15.641138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.668212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 291/574] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:15.669897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.670195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.670483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.689196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:15.690452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:15.758478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6059-fe80-4343-b22d-ac66d32ab79c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.768386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 292/575] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:15.768386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.807152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.807998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.808286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.808596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.816078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.820933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.822414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:15.841996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:15.842939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:15.843868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:15.843868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:15.843868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.843868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:15.843868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 284/576] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:15.844151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.878050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:15.884177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c922d-7841-498a-b189-18d609fd9aa0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.923027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.923835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.924124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.924441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:15.936575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:15.938726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:15.952563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10228-6108-4af6-a648-c39c6a66b8bf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:15.960264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:15.961177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 293/577] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:15.967194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:15.997243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:15.998038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:15.998326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:15.998758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] PATH_INFO: `/v3/users/glance` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.008295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.010038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:16.025497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.026467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:16.027412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 285/578] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.028826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.072255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ac28b-ccbf-4de9-be6a-98b2b3d28b61 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.087786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.112739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.113540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.113830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.114109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.126607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:16.135220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.136149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 294/579] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.142953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.149844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff30b9-528a-4dfb-8d95-223a9e37cfae None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.155080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.185645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.186429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.186719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.186998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.187781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:16.188120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.189578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.202524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 286/580] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:16.203707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.254860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea62d-b614-4db4-abcf-2a76943a04e2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.290899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.291566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.291853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.292131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.292907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:16.293202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.294590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.304883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2b217-f9f8-44c3-b397-1cde57e4aea5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.306436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 295/581] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:16.307258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.321620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.346705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.347414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.347704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.347983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.349222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:16.349490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.351559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.370394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 287/582] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:16.371223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.388005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.425578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f829-58d0-4d0d-867f-e7693c341bc2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.466422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.467110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.467404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.467688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.468479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:16.469389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.470881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.479440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 296/583] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:16.479440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.486413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f3154-b002-4f94-8148-2ec592afc20f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.522438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.523229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.523517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.523822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.531013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.533043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:16.548406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.549367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 288/584] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.550301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.579376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.594682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eed84-59a6-4859-b3a9-141ca3b01c5d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.625848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.632958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.634162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.634523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.634731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.642899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.644295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.658941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040202cd-e08d-4c0b-bde5-bb13747a1ebe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:16.659518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.660402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:16.661316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 297/585] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.661798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.697602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.698406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.698692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.698982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.709614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.711082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:16.729621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.730769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.731780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:16.731780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:16.731780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:16.731780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:16.731780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 289/586] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.737082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.776047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4941f-5057-4da4-9645-0126d5de53d2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.813910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.815268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.815848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.816393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.830377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.831847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.833323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:16.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:16.857509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:16.858446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:16.858446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:16.858446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:16.858446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:16.858446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 298/587] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:16.858742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7057473d-759c-48e6-a09d-7cdeecf2e4e7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:16.858823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.898726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:16.899330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:16.899623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:16.899905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:16.901322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:16.901642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:16.903549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:16.904349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:16.928340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 290/588] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:16.928731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:16.964121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e999571-83a0-4412-bbef-617e83cf0e0d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.000108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.000835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.001107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.001398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.002253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.002494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.003935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.020343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 299/589] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.020343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.051376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745ca69-d99d-446a-a36d-e93e7725383d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.091885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.092588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.092877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.093155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.094571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.094884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.096405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.104344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.114871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 291/590] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.116143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.125551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027afbe-3911-438a-9ed7-aa5a06a4cee7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.127236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.160274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.160891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.161179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.161452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.162181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.162486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.163868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.177451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 300/591] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.182389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.225209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248bc6f-948e-4272-9250-45ed90f27105 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.261335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.262132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.262417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.262695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.274268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.275756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.283077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aeb22d-bfd0-42ce-8ac8-865e2a3aa9cf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:17.294594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:17.295482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 292/592] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:17.297410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.323523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.324445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.324742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.325054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.341475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=71160dfe2635497abc4cd1c1bffc4531, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.342857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.351069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.399865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2077131-5f4f-47c8-823a-1573bf068db2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.432791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.433514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.433798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.434076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.440916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.442447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.454200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 301/593] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:17 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:17.455044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:17.464346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:17.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:17.473948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:17.475087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 293/594] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:17.476227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.482044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.584503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970a20f-83c7-48ae-a085-632b326c1db5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.594744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a181a54-967e-4036-9e0a-fef59ef515ef None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.607344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.623926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.625212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.625681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.626126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.634683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.635514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.635907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.636026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.636449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.637282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.637695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.638430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.639256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.653730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 294/595] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.654714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:17.659403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:17.660701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:17.661849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:17.664088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 302/596] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:17.665061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.746223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.760508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398854e-8984-4230-bc5e-dc31caf36eb0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.778752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592ff36-8a83-47ca-b22f-d84e1089850c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.794564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.795221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.795513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.795793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.796619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.799918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.799918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.810864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 295/597] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.811774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.823878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.825089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.825655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.826081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.834252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.836056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.845041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-470d85d4-63d0-49a4-9073-0df6f6a04973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:17.852534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:17.859876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 303/598] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dea68-c58d-4074-868d-2824395f56dd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.860842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dea68-c58d-4074-868d-2824395f56dd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.861427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dea68-c58d-4074-868d-2824395f56dd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.861427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 304/599] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:17.861427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dea68-c58d-4074-868d-2824395f56dd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:17.922686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea587-6d6e-42e6-ad38-5f7bbece62a0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.962547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:17.963219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:17.963504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:17.963829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:17.964655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:17.964964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:17.966405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:17.973643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dea68-c58d-4074-868d-2824395f56dd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:17.985207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 296/600] 10.222.0.47 () {56 vars in 1139 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:17.986907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.006643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.011466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.012379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.012662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.012946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/b32ef598390145d1b88c3e3811edb79b/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.027966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=b32ef598390145d1b88c3e3811edb79b, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.029602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bc5f9-3e23-4ca1-83d2-187ea256c9f7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.081055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.092274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e3c59-0a21-4939-ab67-4ec8068f15ec None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.116344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 305/601] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:17 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/b32ef598390145d1b88c3e3811edb79b/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:18.116344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.116344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.116344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.134939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.135893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.136443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.137152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.137987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:18.138352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.140030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.142979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:18.145162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:18.155146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 297/602] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:18.156058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.236826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 306/603] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:18.237735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.307042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38510de-9df9-404f-a728-5ea12bb42879 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.354741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.355604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.355894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.356171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.370425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a2cc7-9a61-4baa-a311-8cb9d6134aed None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.371562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.374213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.384238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.385714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:18.397847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:18.412804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 298/604] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:18.413686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.415845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.417284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.417582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.417912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/44fe9acbbbdc463ab22869886df4e3aa/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.439559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=44fe9acbbbdc463ab22869886df4e3aa, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.443640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.523523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae3a21-6724-4648-9ab7-5d0b92511718 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.542146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 307/605] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:18 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/44fe9acbbbdc463ab22869886df4e3aa/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:18.543112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.571998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.573488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.578528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.578528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.589994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:18.600719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:18.602170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:18.603278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 299/606] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:18.613025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.742412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e5dc9-b293-4ae9-a201-94f7f17e6a71 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.752481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.798125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.798812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.799108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.799393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.801947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:18.802286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.804359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.809697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ddaad-eefc-4cc8-abd5-2b64d319e47c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:18.817660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 308/607] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:18.823244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:18.857180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:18.857861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:18.858211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:18.858494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:18.858595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:18.859341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:18.859642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:18.861099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:18.870137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 300/608] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:18.876090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.016610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824dbb6-a674-4f81-84a7-ffafa8e99199 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.035112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e25eb-638a-4213-9a5b-31ea9cec8dd8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.040884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:19.059950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.060784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.061078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.061355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.085783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.087180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.102732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.103539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.103827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.104099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:19.121131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:19.122065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:19.122966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:19.122966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:19.122966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:19.122966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:19.123610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.128464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 301/609] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:19.129327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.133675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:19.144999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:19.145927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:19.146812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 309/610] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:19.153414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.315265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69c760-905f-4774-9a87-a20f3146ad4b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.337659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:19.360369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.361436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.361937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.362512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.363486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:19.364186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.366931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.372813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05caad-3f76-453e-812a-1f2cccdeda82 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.392515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 302/611] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:19.395852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.442828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.443507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.443796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.444076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.444926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:19.445237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.446696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.448543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:19.479906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 310/612] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:19.498323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.524647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6b6c9-9e49-407f-8fe3-7b47a8456378 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.561316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.563027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.563533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.564035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.573244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.575695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9741e533-f604-4b7f-806e-b0f6bfd8389b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:19.602701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:19.604497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 303/613] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504cf7-d3b1-48f4-b28a-4ad6e5563f6f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.605642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504cf7-d3b1-48f4-b28a-4ad6e5563f6f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.606527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504cf7-d3b1-48f4-b28a-4ad6e5563f6f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.611431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 304/614] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:19.611431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23504cf7-d3b1-48f4-b28a-4ad6e5563f6f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.668611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00336506-d279-456c-8c27-53da41d022b1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.712576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.712780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:19.714023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.714818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.715549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/bd67da2713214172989b5c689a8885f4/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.721509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23504cf7-d3b1-48f4-b28a-4ad6e5563f6f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.730613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:19.735417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=bd67da2713214172989b5c689a8885f4, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.737708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.757432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.758208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.758593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.758926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.759848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:19.760228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:19.761776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b49705-9afa-4e1b-91db-c459541eb61a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:19.784374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 305/615] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:19.784374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.784374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.784374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:19.799875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:19.801096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:19.807786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 311/616] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:19 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/bd67da2713214172989b5c689a8885f4/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:19.809454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:19.939255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ebc47-ab19-4abf-a1e2-cd2b5eed3a3b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:19.986110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:19.987577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:19.988077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:19.988632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.001974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.003825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.014116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 306/617] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:20.034374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:20.035370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:20.036433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:20.036433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:20.036433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:20.036433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:20.037938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 312/618] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:20.043778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.063643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.128742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.169389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.173611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ce3b-878a-46a6-9d8f-c3feb0b22160 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.211344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.212080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.212390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.212675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.222276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e77ae4-2301-48a3-bfcc-4286045a6e6d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.224881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:20.233655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:20.235190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:20.236019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 313/619] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:20.241982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.351682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec59f6-ce9b-4648-a621-957d09696299 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.364689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.387654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.388457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.388819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.389192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.389965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:20.390275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.392192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.401054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.401754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.401871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 314/620] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:20.402049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.402341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.403096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:20.403414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.404883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4fef7-2501-4c8e-af56-c3bc9edd7002 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.407678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.452289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 307/621] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:20.457582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6e34-adb0-4bac-b21a-27dcfca8fdfc None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.458124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6e34-adb0-4bac-b21a-27dcfca8fdfc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.458587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6e34-adb0-4bac-b21a-27dcfca8fdfc None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.460329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 308/622] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:20.461250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc6e34-adb0-4bac-b21a-27dcfca8fdfc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.524415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6035730-7345-48b4-8f0c-3c1b76463e82 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.565940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.567132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.567724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.568376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.579865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.581965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.588002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc6e34-adb0-4bac-b21a-27dcfca8fdfc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4f9bc4d0-8739-4b5d-9108-8fcffd319506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:20.602134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:20.603061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:20.603942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:20.603942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:20.603942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:20.603942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:20.606490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 315/623] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:20.607965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.608495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.609056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.630404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.631335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.631626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.631908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.632843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:20.634350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:20.639529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.641467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.649926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:20.656117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:20.657141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:20.658021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 309/624] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:20.658973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.663909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.720379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 316/625] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:20.721965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.786029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b0dd4-aac6-4c3b-bf08-76b32506cba6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.834732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.835774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.835908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.836250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.837087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:20.837515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.839043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.852687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed20eb0-6a3a-4be8-8d24-46f97e111046 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:20.858139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 310/626] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:20.859262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.889270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:20.890251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:20.890972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:20.891454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:20.892484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:20.893018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:20.894657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:20.908171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 317/627] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:20.909347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:20.921064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.931206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:20.973054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a2c10-8c9b-406d-8c4e-363fe9121320 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.011361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.012461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.012755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.013038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.015823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73feef-0705-48d2-b8ce-a9f34739e029 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.024244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:21.032572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:21.033524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:21.034419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:21.036522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 311/628] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:21.037257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.055843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.056680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.056968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.057246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.065194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.066660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f15694f9-1668-4ec7-89fa-b15cacd2badd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:21.082114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:21.083180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:21.084433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:21.087930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 318/629] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:21.089252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971915-3d5b-40ec-ad25-0951d12ba52f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.089544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971915-3d5b-40ec-ad25-0951d12ba52f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.089824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13971915-3d5b-40ec-ad25-0951d12ba52f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.092347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 319/630] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:21.092347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13971915-3d5b-40ec-ad25-0951d12ba52f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.157686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f82e9-662e-43ed-8fe6-e5b8a61dd21a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.173361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.180419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.192063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.193033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.193320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.193595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/fe3f01883b804652b396cd4e28ab4b46/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.208517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=fe3f01883b804652b396cd4e28ab4b46, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.209969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18541f5b-c2d3-44f7-81df-ab90938c0c7b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.210484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13971915-3d5b-40ec-ad25-0951d12ba52f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.244646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.245333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.245623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.245905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.246660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:21.246969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.248355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.256438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 320/631] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:21.257221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.300420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 312/632] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:21 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/fe3f01883b804652b396cd4e28ab4b46/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:21.300420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949031e6-5200-460a-bde0-383434a275f3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.300420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949031e6-5200-460a-bde0-383434a275f3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.300420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949031e6-5200-460a-bde0-383434a275f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.318636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-949031e6-5200-460a-bde0-383434a275f3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:21.321960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-949031e6-5200-460a-bde0-383434a275f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:21.382909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f483e-cd88-464c-a09e-230d4c9ee4fb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.404778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 313/633] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:21.406652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949031e6-5200-460a-bde0-383434a275f3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.429360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.440432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.441961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.441961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.441961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.445202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:21.445527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.447034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.475259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 321/634] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:21.476501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.554159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949031e6-5200-460a-bde0-383434a275f3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.555731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.589834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.590618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.590954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.591237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.601378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b50251-1732-48be-9d76-7a7819b06ea0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.602589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.604184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:21.623680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:21.626320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 314/635] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:21.627373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.643050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.644017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.644443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.644826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.659009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:21.667740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:21.668904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:21.669907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:21.673297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 322/636] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:21.677914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.734916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.736590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a805313-9c1e-42a4-9200-91450fe59d53 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.773126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.773745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.774034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.774317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.797335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45aacbe-5698-40e8-91e4-a1c4dc83d874 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.799943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.843345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:21.843965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:21.844848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:21.844960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:21.845808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:21.848491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.848491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.866746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 323/637] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:21.867318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:21.869487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:21.872453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:21.971516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df667b21-9942-4a29-960b-217f28efd321 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:21.981665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:21.988259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 315/638] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:21.991304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.014662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.015358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.015642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.015926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.016885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:22.017200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.018669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.031800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 324/639] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:22.031800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.057064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.106163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701108bc-d665-4be1-9a16-b5abc5ee248c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.135876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7b403-4d04-4875-b9e4-d1fdeb4713dc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.144193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.145091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.145374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.145649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.153227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.154668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:22.169201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:22.170163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 316/640] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:22.171087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.177857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.178902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.179269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.179550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.190573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.192017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f1e7cd02-5fcb-4241-a7d5-c709792c1a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:22.209756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:22.210934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:22.211852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:22.211852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:22.211852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:22.211852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:22.211852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 325/641] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:22.212143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab169-f2cd-4f53-9060-9cee5f8be115 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.212582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab169-f2cd-4f53-9060-9cee5f8be115 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.212746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceab169-f2cd-4f53-9060-9cee5f8be115 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.216432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 326/642] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:22.216432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab169-f2cd-4f53-9060-9cee5f8be115 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.241127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.284558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1a7ed-f0f3-41cf-8cb3-2bc3c2377839 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.298782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.319810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.320382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.320670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.321003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.324645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceab169-f2cd-4f53-9060-9cee5f8be115 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.332634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.334260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:22.351978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:22.353119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 317/643] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:22.354122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.367633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.368250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.368567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.368853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.369713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:22.370030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.371499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b176a6c-d5e7-44ed-b5d0-95e2e91a0d49 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.393175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 327/644] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:22.394724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.395149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.395560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.415043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:22.416252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:22.475484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e62cbd-b80f-438f-b4fd-d5dd52eba9f4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.494174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 328/645] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:22.495298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.510687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.514204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.515399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.515964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.516806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.517556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:22.518066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.519473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.532264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 318/646] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:22.533095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.563135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.608669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21115ccc-761d-4f85-a685-6fea47d7f679 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.638357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62459e-0cb4-48d1-aef1-b753f5c89245 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.646150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.646845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.647135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.647416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.648339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:22.648461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.650002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.665978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 329/647] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:22.665978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.678901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.679632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.679988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.680280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.739028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.751144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.752689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:22.770540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52ebc0-9978-4e4d-9e82-8f39835ac547 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.780324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.808097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.809013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.809308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.809638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.820105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:22.828514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:22.829496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 330/648] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:22.835236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.864867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 319/649] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:22.865857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:22.941779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aab82-f2e1-4aba-9722-11d1c283b2c9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.971915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:22.975380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01371e26-719e-4dc3-b239-07be08c2bb28 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:22.979449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:22.980247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:22.980612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:22.980918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:22.989915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:22.991369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.002110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:23.005960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:23.007039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:23.008116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:23.012817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.018689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.018689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.018689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.018960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 331/650] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:23.018960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.022594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.024132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:23.040203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:23.041398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:23.042485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 320/651] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:23.043231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.132531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796de6b-a374-41d9-b935-fb9f99cbfb25 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.150702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1036f-9e30-4f22-8901-67561f6289a2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.169302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.169982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.170266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.170544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.171301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.172191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.173612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.186440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 332/652] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.186440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.191190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.192047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.192283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.192589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.205264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.206740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:23.225629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:23.226634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:23.227524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:23.227524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:23.227524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:23.227524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:23.227524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 321/653] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:23.227812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.242980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.271032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.291770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f727-3aff-4f95-8a42-633c8ca95bca None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.329059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.330135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.330601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.331058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.332000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.332502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.333944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.342800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e2e99-273e-4edb-b288-966db4a4931b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.352201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 333/654] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.352201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.379268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.379950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.380239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.380599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.381399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.381714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.383165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.401258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 322/655] 10.222.0.47 () {56 vars in 1137 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.401895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.459715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8be268-479f-4ad5-9840-154a0fbdbc5b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.472716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.494465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.495265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.495560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.495834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.500300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.506629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.507326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875e52-00e9-4aae-b938-d5cca0de18cc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.508082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:23.525525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:23.526500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:23.527414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:23.527414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:23.527414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:23.527414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:23.527414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 334/656] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:23.527702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.545417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.546086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.546379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.547183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.547905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.548214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.549616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.565318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 323/657] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.565318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.626829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b6d1e-5b4e-4605-b38a-843cbd0dd652 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.659705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f026e43-ab36-40b7-a099-a976283a1580 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.660595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.661252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.661542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.661825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.662675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.662994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.664459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d3ee86-59a4-4851-bff9-3b30ef076ec5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.670191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 335/658] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.671393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b808e3-ebb3-4e6d-8680-9dfaa1e10970 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.672221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b808e3-ebb3-4e6d-8680-9dfaa1e10970 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.672533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b808e3-ebb3-4e6d-8680-9dfaa1e10970 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.673767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 336/659] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:23.674487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b808e3-ebb3-4e6d-8680-9dfaa1e10970 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.706612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.717773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.718973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.719258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.719539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.750115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.791499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b808e3-ebb3-4e6d-8680-9dfaa1e10970 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:23.801758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.803383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7b772-80d5-4c68-a658-f1251186fd35 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.836629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:23.837294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.840181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.840181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.840181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:23.840181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:23.840587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:23.853897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 337/660] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:23.854610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:23.925524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 324/661] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:23.926800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:23.927163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:23.927516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:23.945291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:23.946481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:23.963198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:23.976595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41191ce-73c8-4df9-8e18-a99ba7587f03 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.002188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.014700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.015746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.016061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.016343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.028363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 325/662] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:24.028363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.028605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.028605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:24.040750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:24.041687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:24.042597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 338/663] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:24.043272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.141630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a06ee-79c4-41b9-8236-033005b776fe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.144793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a439f1-3487-45ed-8808-0e97e8089d36 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.178248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.178480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.179050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.179264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.179378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.179546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.179641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.179818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.187126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.188656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.191460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.192950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.201070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:24.205549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:24.206636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 18:06:24.207765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:24.209974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 326/664] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:24.211072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:24.211186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:24.211985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:24.215219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:24.215219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:24.215219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:24.215219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:24.215219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 339/665] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:24.216741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.264481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.318569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63437eb7-67b7-4408-9b64-abd5bdf4f1cd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.323498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89960-5640-4cde-a425-5891028a7e93 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.354795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.355870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.356159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.356472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.362933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.363598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.363891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.364173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.365006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:24.365406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.366753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.368046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:24.376556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:24.377503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:24.378347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 327/666] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:24.380089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.390737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 340/667] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:24.390737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.459598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.494824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd90af-f0b8-41fa-a365-70347517faf2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.502458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.513148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f313-d950-406a-bbd9-b06550d9fde4 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.531368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.532008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.532293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.532605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.533451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:24.533763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.535357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.548730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.549399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.549689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.549971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.550733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:24.551044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.552458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.559808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 328/668] 10.222.0.47 () {56 vars in 1139 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:24.559808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.565604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 341/669] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:24.566372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.665372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143f1bf-5d03-4469-9853-4e1a48e69d1f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.669410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bbacb-5771-45fe-8c2c-35c8bff4ea52 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.693305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.701234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.701978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.702195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.702464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.703769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:24.704086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.705496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.713820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.713820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.713820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.713820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.714155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=b32ef598390145d1b88c3e3811edb79b) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:24.714155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.714155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.714361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 329/670] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:24.714877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.732340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 342/671] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/role_assignments?user.id=b32ef598390145d1b88c3e3811edb79b&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:24.732340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.756920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:24.854487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd07f1-a321-4f8a-a1b6-c197af143bc3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.861344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef77e-4922-411c-a8a3-a55202133a78 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:24.893070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.893880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.894169 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.894449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.902055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:24.903539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:24.909884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:24.911212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:24.911212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:24.911493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:24.918794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:24.919857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:24.921248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 343/672] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:24.922352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:24.979421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.012083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.014364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.051462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.055271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56d746-a06a-4c75-b35f-7361724cbe52 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.097524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.098845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.099202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.100258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.113786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.115946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.135153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 330/673] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:25.136121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:25.143025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:25.144077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:25.145212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:25.145212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:25.145212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:25.145212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:25.147055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 344/674] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:25.148678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.272124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.278000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6c794-8f51-49f9-9802-bccda9a58459 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.283434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6443e-feba-4cd3-951e-d74732c63b3f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.308885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.323431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.324284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.324702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.325082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.326198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:25.326609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.327267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.328358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.328481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.328933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.329419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.341775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.344065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.344898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 331/675] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:25.345702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:25.364642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:25.365596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:25.366471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:25.366471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:25.366471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:25.366471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:25.368728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 345/676] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:25.369871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.467091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397b43e-1599-48af-906e-1ea45715d083 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.487054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab217-d374-4608-ab45-220eefad796c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.505505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.506315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.506603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.506880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.507711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:25.508020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.509443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.522244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 332/677] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:25.523086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.526376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.527384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.527879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.528412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.529466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=44fe9acbbbdc463ab22869886df4e3aa) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:25.530007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.532257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.539765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 346/678] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/role_assignments?user.id=44fe9acbbbdc463ab22869886df4e3aa&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:25.540900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.559225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.575429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.636188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1aeb4-16de-4f29-b8cf-05a1ed3e920f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.665241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd8591-0bb6-46f0-84cd-3adabf17a0b8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.682592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.683446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.683755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.684142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.684947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:25.685257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.688072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.705209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 333/679] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:25.707326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.708203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.708599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.708954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.710310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.717167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.718789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:25.735601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:25.736667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:25.738154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 347/680] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:25.744246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.833559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.866116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12e12-f82a-4fe4-aa94-8ae5386b049b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.873348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605edee-3877-4b6d-b07e-6738b1381916 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:25.876601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:25.922450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.923728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.923991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:25.924516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.924710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.925009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.925299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.925596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.926164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:25.926497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.927953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652a38a-e76d-43fb-bbd2-0b4a86e6b2bd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.934878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:25.937144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:25.948069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 348/681] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:25.949519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830a8d6-ccea-44e9-8b13-4802b366decc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.949821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830a8d6-ccea-44e9-8b13-4802b366decc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.950112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830a8d6-ccea-44e9-8b13-4802b366decc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.951632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 349/682] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:25.956616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830a8d6-ccea-44e9-8b13-4802b366decc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-bbefd104-dd3c-47ce-82ac-9db155d06685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:25.959131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:25.959993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:25.960920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 334/683] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:25.962074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:25.962379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:25.962723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:25.981614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:25.982900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:26.064330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 335/684] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:26.064330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.105356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830a8d6-ccea-44e9-8b13-4802b366decc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.142669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.149111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.149934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.150354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.150744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.196992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.205308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35a3dc-2702-4dca-a11d-85e1ffd916e0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.238066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.239628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.245548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.246435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.246868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.247352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.248452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:26.248912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.251326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.264691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 336/685] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:26.266738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.356076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 350/686] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 18:06:26.356998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.374048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570e10f-3783-4253-98ff-2d1d454e8898 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.396032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.421531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.423462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.423970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.424506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.440678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:26.451791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:26.453603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:26.454935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:26.458625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 337/687] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:26.459680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.486124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.490431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb509c-db74-421b-8c49-9f1a9ee3e848 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.526591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.527621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.528137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.528652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.529704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=bd67da2713214172989b5c689a8885f4) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:26.530249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.532163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.544579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 351/688] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/role_assignments?user.id=bd67da2713214172989b5c689a8885f4&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:26.545875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.576781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22f244-121e-460d-9805-340aff5bce40 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.616660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.617448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.617737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.618012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.632345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.632345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:26.651377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:26.652386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:26.653313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:26.653313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:26.653313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:26.653313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:26.653313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 338/689] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:26.662719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.704757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.711817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273616a-7f44-42d2-8245-64b5ef77aee9 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.792731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.793415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.793695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.793970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.794729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:26.795036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.796521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.798327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:26.814879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 352/690] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:26.823008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.853608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f1f07-a8df-42ba-b1bb-2036b44d218c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.895868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.897538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.897830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.898111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:26.898852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:26.899155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:26.900589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:26.922129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 339/691] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:26.925487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:26.945788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966537-d82b-462d-8a09-b5a3090340ec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:26.987391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:26.988183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:26.988497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:26.988774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.009216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.012047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:27.045614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:27.047323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:27.048207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:27.048207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:27.048207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:27.048207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:27.054325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 353/692] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:27.061630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.080885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096ac55-72d8-4147-86da-c99c41a3f8f2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.083959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.128925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.157686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.158421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.158707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.158982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.173815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.175587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:27.204608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:27.205583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:27.206486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:27.211325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 340/693] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:27.220863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.248565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a9e1b-4150-43e4-866b-b9cfc8227898 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.293134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.294363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.295148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.295644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.296679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:27.297223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.299539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949212e-1096-4410-be76-370b7f10e3a1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.320722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 354/694] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:27.322654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bb746-7775-4f1f-82a7-c40ad3b8b709 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.324001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bb746-7775-4f1f-82a7-c40ad3b8b709 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.324531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bb746-7775-4f1f-82a7-c40ad3b8b709 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.326318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 355/695] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:27.327255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bb746-7775-4f1f-82a7-c40ad3b8b709 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.369032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9259-240c-4f29-a810-5e36859340a6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.412885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.413060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.414478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.414787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.415078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.416232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:27.416684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.418360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125e470-0c4b-46bc-b6df-01e9dd9d50d1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.441680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 341/696] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:27.441680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.441680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.441680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.466652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:27.467982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:27.468098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bb746-7775-4f1f-82a7-c40ad3b8b709 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.507749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.509083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.509646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.510145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.517170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.520911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.522688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:27.539431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:27.540585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:27.541723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:27.544800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 356/697] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:27.546027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.550409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 342/698] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:27.551364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.678950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09fed4b-5ee3-4ced-8641-6da4fd5691f2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.691764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39747a50-0c61-42bb-ab31-a9e1c2a41d27 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.707555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.719494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=fe3f01883b804652b396cd4e28ab4b46) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.729976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.735298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 357/699] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/role_assignments?user.id=fe3f01883b804652b396cd4e28ab4b46&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:27.736530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.746644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.747454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.747636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.747908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.749060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:27.749174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.751575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.810732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 343/700] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:27.816806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:27.882259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b814c3-f897-4916-9b54-de77726dcca6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:27.903299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:27.943013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:27.944677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.945664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.946179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.959928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:27.961742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-654be5a6-c38e-4eca-aacf-fc39097b13c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:27.982729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:27.983804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:27.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:27.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:27.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:27.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:27.989803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 358/701] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:27.991292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39c42-2274-4489-913e-1367e9045a4c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:27.991497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39c42-2274-4489-913e-1367e9045a4c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:27.991784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb39c42-2274-4489-913e-1367e9045a4c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:27.993666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 359/702] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:27.994312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb39c42-2274-4489-913e-1367e9045a4c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.001488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c231-ad8c-4358-83b4-96ac1c2bf268 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.043503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.044164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.044493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.044774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.061945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.122279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb39c42-2274-4489-913e-1367e9045a4c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.136981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.138991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.159966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.160931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.161363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.161836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.162785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:28.163285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.164824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422b68d-3be0-4f0b-85bc-afb15d80d06a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.183180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 360/703] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:28.184758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.184898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.185324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.185827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.203902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:28.205446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:28.269896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 344/704] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 18:06:28.271201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.295058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 361/705] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:28.295959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.349074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.412659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87869d8-555a-4852-a823-35274b2d71a5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.425572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d78f6e-6dad-4e88-bb6e-99a28308583b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.450777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.451470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.451761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.452042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.453029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.454847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.463076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.473846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.474662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.474965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.475234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.487432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.488944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:28.507545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:28.508515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:28.509447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:28.509447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:28.509447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:28.509447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:28.509447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 362/706] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:28.514634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.563364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 345/707] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:06:28 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:28.565273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.697273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.702835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583fc3b-4bf4-419e-8a2b-65773f9fb0a1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.725789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3381a4-92c5-4366-937b-e42a49609b2f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.764820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.765875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.766165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.766449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.767500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:28.767812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.769227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.771134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.772434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.772726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.773014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.773687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.775060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.782704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 346/708] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:28.788039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:28.794843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:28.802464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:28.803416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 363/709] 10.222.0.47 () {56 vars in 1129 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:28.804266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.859304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:28.911853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635d09-e65c-4a91-98c8-198e4957409a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.912685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5dc08-461f-4521-b2af-17b6477cc108 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:28.949242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.949914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.950274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.950534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.951307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:28.951616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.952413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:28.953664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.954275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:28.954886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:28.955310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:28.962888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:28.964755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:28.967840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 364/710] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:28.977601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:28.981858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:28.982720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:28.983602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:28.985690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 347/711] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:28.992568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:29.002350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:29.119648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629effa7-d57c-4535-8976-742fd308cc84 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:29.132905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e94b115-9751-4d6c-918b-f48d517fa5af None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:29.134991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:29.159468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:29.160440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.160845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.161231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.162172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:29.162668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:29.164412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dc0f-a243-409d-86de-ecdd07139e62 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:29.172036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:29.173069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.173566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.174020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.174912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:29.175387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:29.176991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:29.178835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 348/712] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:29.180252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0b17a-1a70-455e-beeb-5d78bafa50f2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.182488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0b17a-1a70-455e-beeb-5d78bafa50f2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.183140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0b17a-1a70-455e-beeb-5d78bafa50f2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.185547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 365/713] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:29.186642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:29.188482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 349/714] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:29.191441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.191741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.192023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.213534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:29.214807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:29.324528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:29.326323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 350/715] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:29.350778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067dd30-f5e9-4969-8cb6-a728e088d1d5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:29.361517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:29.426624 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:29.427305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.427594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.427983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.428991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:29.429316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:29.430872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:29.438947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 366/716] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:29 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:29.505639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2fd8d-37b8-4921-9404-2a1d1864f5d5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:29.507749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:29.545091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:29.545918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.546201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.547152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] PATH_INFO: `/v3/services/network` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.548047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:29.549553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:29.565278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:29.566376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:29.567489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 351/717] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:29 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:29.573308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:29.830110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f9ece-7e59-4bfa-9d4f-1903f13269e0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:29.876706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:29.877707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:29.878177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:29.878626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:29.879513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] RBAC: Adding query filter params (name=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:29.880004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:29.881619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab6af-4601-4909-8337-bc8dd6de7182 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:29.888575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 367/718] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:29.897395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:30.041035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb83371-323e-4343-880e-fdc4d720f48f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:30.090957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:30.091547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:30.091831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:30.092108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:30.092849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] RBAC: Adding query filter params (type=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:30.093175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:30.094569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403a76-83a4-452c-8893-6f2bb1df5799 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:30.105325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 352/719] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:30.213427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eef2d-3bc1-4417-a937-3dd1114f1c2a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:30.214411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eef2d-3bc1-4417-a937-3dd1114f1c2a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:30.214891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90eef2d-3bc1-4417-a937-3dd1114f1c2a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:30.216833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 368/720] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:30.222707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:30.223009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:30.223528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:30.245051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:30.246275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:30.281106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:30.340145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 353/721] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:30.367898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eef2d-3bc1-4417-a937-3dd1114f1c2a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:30.378326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:30.494838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90eef2d-3bc1-4417-a937-3dd1114f1c2a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:30.531942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:30.533130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:30.533617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:30.534094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:30.548861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:30.556699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:30.557712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:30.558587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:30.565058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 369/722] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:30.573398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:30.720268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c6515-c8cd-49e5-b8ce-ad94991ec980 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:30.770367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:30.771038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:30.771323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:30.771597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:30.772366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:30.772675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:30.774052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:30.788048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 354/723] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:30.796935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:30.894907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:30.951073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d067f4a-3981-4d0d-bc2a-46b9f678d340 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.004068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.004826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.005117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.005401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.019926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:31.021437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:31.045431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:31.046431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:31.048145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:31.048145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:31.048145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:31.048145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:31.048145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 370/724] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:31.053456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:31.100708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:31.210636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff108597-3cab-4a69-afc1-6ff392ec52d4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.266789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.267432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.267720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.267997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.268759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:31.269070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:31.270449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:31.287610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 355/725] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:31.292761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:31.443717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807229cb-6801-4103-b9a7-27e795bc5f14 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.462331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:31.476353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.477206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.477495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.477779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.492875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:31.492875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b78bae89-334e-410c-ac2b-7604fe13ce33 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:31.502718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:31.503666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:31.505105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 371/726] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:31.513429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:31.527618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adacdd-7abd-4fee-a439-9abe5fb267d3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.527903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adacdd-7abd-4fee-a439-9abe5fb267d3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.528085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adacdd-7abd-4fee-a439-9abe5fb267d3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.529505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 372/727] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:31.539696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a594ee-a8a8-450f-a98a-0012183f1a27 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.540022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a594ee-a8a8-450f-a98a-0012183f1a27 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.540245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a594ee-a8a8-450f-a98a-0012183f1a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.563268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-93a594ee-a8a8-450f-a98a-0012183f1a27 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:31.564527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-93a594ee-a8a8-450f-a98a-0012183f1a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:31.631981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:31.652229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 373/728] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 18:06:31.654530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1603-e208-4c26-8805-a39ee6139bcc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.654978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1603-e208-4c26-8805-a39ee6139bcc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.655407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1603-e208-4c26-8805-a39ee6139bcc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.657691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 374/729] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:31.665174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.665821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.667820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721923a-6331-4801-8633-e5aebb14116e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.669777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.694783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:31.696238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:31.717860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.719417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.719716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.720003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.721055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:31.721508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:31.723510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:31.756919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 356/730] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:31.756919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:31.787446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 375/731] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:31.788869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:31.863826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:31.890150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121219ab-92dd-4961-92aa-797c5f46962f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.928518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.928809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.929086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:31.929810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:31.930053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73ddc7-27d2-4bed-a9b3-1dcad31d9ae7 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:31.931200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:31.967394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:31.968943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:31.969381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:31.969811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.030085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.044617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 357/732] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:31 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:32.044617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.065714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.069519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608f7aa-d194-4312-8c7e-2d9cdfcb006b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.141598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.212641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdc3c1-7dd4-4ab6-bc7f-5633585586ce None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.227595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 376/733] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 18:06:32.229070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff83f-d804-4165-b947-64d2be40a62d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.229362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff83f-d804-4165-b947-64d2be40a62d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.229653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff83f-d804-4165-b947-64d2be40a62d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.231054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 377/734] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:32.235787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff83f-d804-4165-b947-64d2be40a62d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.254648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.255624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.255959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.256242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.270620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-26477749-dada-48b2-a45e-4416c93c3d87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:32.278935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:32.279934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 358/735] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:32.282592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.283351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.283351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.303850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:32.305012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:32.368725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff83f-d804-4165-b947-64d2be40a62d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.377089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.400529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 359/736] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:32.401915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.418051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.419126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.419424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.419828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.434342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.453755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.455407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:32.474969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:32.476087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:32.477249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:32.477249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:32.477249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:32.477249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:32.478771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 378/737] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:32.483463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.533190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f1b19-6ac9-4d0c-8f41-c30b0a49fb6d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.578061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.579604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.580059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.580535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.581621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:32.582369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.584374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.587764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9736c72-b2be-4fed-aa37-d29b37772ac7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.596816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 360/738] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:32.601660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.630501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.631146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.631432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.631706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.643044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:32.652418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:32.653384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:32.654242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:32.659080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 379/739] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:32.659975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.670060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.712934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.733707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b0ec3-0421-4e5b-8597-b78261a5fa58 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.767079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25639e-4831-4b83-86bd-53799e713293 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.768961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.769637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.769922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.770196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.770924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:32.771229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.772649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.786006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 361/740] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:32.789238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.804528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.805451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.805854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.806143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.819097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.820616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-60144fbc-56be-4f99-8157-fabfc3c1b8a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:32.839922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:32.841010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 380/741] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e164d1-7780-468c-8a0c-f71524f3d5e9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e164d1-7780-468c-8a0c-f71524f3d5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e164d1-7780-468c-8a0c-f71524f3d5e9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.846886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 381/742] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:32.848660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e164d1-7780-468c-8a0c-f71524f3d5e9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:32.931029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bb837-7203-4159-a208-b79f60fa6997 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.941634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.967793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e164d1-7780-468c-8a0c-f71524f3d5e9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:32.967996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:32.969036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:32.969345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:32.969633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:32.969983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:32.971424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:32.971562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:32.973614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:32.986957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 362/743] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:32.986957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.009905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.010703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.010995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.011280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.018497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.019954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f7fa48e2-327a-41fd-b023-31e75015bb66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:33.035122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:33.036003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:33.037516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 382/744] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:33.038003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.038071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.038417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.055517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:33.056752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:33.097306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc3860-4435-463d-9ce7-b91b723cd854 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 383/745] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 18:06:33.129250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.139074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.139744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.140692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.141115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.142108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:33.142481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.143878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.156615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 363/746] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:33.157513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.190238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.235728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.245673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba527-baf0-43e0-bf36-783f363de745 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.271112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd72b71-6375-42b4-90d1-4bdd73fae4b5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.284008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.284841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.285129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.285406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.296491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.298047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.308883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.309823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.310409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.310885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.311932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:33.312499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.314198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:33.316272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:33.317168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 384/747] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:33.322708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.333884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 364/748] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:33.333884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.428514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.439889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e552e-8620-48a0-88b5-fbb0a25a69b8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.449650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd683585-0b12-4d32-b335-adafeae1dcf2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.477977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.478789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.479074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.479356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.486538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.487200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.487485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.487765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.488166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.488514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.489686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.489866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:33.504900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:33.506046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 365/749] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:33.507000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.559690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 385/750] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:33 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:33.561020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.570154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.629351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6138d46-0c4c-43b0-a7da-d9f18cb22812 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.710746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb47d3-bcad-4ec5-afed-61bb28544fde None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.754579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.755545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.755991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.756452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.757484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=fe3f01883b804652b396cd4e28ab4b46) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:33.757953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.759581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.767430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 386/751] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/role_assignments?user.id=fe3f01883b804652b396cd4e28ab4b46&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:33.768233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.815339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.816027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.817104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.817454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.818221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:33.818531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.820922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.839972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 366/752] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:33.847837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.849044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.863501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:33.892126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1180d-8321-45a2-a3cb-47a2942c143b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:33.932280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:33.933008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:33.933301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:33.933585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:33.934549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:33.934864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:33.936454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:33.960970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 387/753] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:33.967303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:33.969231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60ca82-b9da-4506-8cbc-2e09817cb09b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.007954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.008943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.009403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.009739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.018543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.020159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:34.040390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:34.041530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:34.042616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:34.044856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 367/754] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:34.051541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.091113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc48ed-6d79-4290-9a9d-9ea195d6a88c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.118794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.138397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.139063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.139398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.139629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.180660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.247059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a9bb9-22a4-4552-947f-0104d36930a6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.257878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.259737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997527-5cdd-4e61-b788-d467de78d156 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.288422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.289414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.290122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.290414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.291553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:34.292115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.294294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa799f-293c-4706-85fd-86a0aa6c5d7e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.318773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 368/755] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:34.320797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51385b7-46f0-4108-b9b3-d14f9412adbd None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.321090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51385b7-46f0-4108-b9b3-d14f9412adbd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.321371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51385b7-46f0-4108-b9b3-d14f9412adbd None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.322853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 369/756] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:34.329873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51385b7-46f0-4108-b9b3-d14f9412adbd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.396984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 388/757] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:06:34.398390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.398659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.398938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.417084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:34.418320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:34.436365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.462416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51385b7-46f0-4108-b9b3-d14f9412adbd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.493980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.512387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.512578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 389/758] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:34.513002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.513002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.513192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.513277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.533753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.535496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:34.562734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:34.563674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:34.564603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:34.564603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:34.564603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:34.564603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:34.566627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 370/759] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:34.567677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.623988 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d724e-3c90-44fa-9fa4-091e7c651cc0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.670951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.673214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.673661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.674095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.687462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.689670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.706250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858310d8-25b5-4174-87ba-0a7951845efe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:34.712705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:34.713793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:34.714929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:34.714929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:34.714929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:34.714929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:34.717291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 390/760] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:34.718839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.724545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.756043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.757769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.758161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.758543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.759495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.761849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:34.777580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:34.778698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:34.779740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:34.787873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 371/761] 10.222.0.47 () {56 vars in 1129 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:34.789063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.839099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:34.842407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c698f1f-ecdb-48c9-b05e-b6419c256d75 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.881430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.882277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.882696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.883195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.884041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:34.884575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.886063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.901180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0d48b-2897-452a-be6f-96a635fa1791 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:34.906959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 391/762] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:34.907980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.940147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:34.941117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:34.941664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:34.942244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:34.943117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:34.943578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:34.945198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:34.959428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 372/763] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:34.960645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:34.989033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.028027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93fcb-1b27-498a-99a2-f80930cbe788 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.069244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.069924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.070217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.070496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.071221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.071530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.072944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.079330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 392/764] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.080147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.086763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df801a-62e1-470e-b4c7-45a2c080a750 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.089713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.124709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.125734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.126087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.126434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.134128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.135970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:35.150785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:35.151690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:35.152636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 373/765] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:35.153897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.201553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3a140-be9a-4e65-aca2-d8aa50a1015f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.237326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.247512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.249024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.249997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.250533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.259644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.262060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:35.281906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:35.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:35.284112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 393/766] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:35.284640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.288051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340de3f8-7394-4f94-9956-ee81304e1abc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.335891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.336739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.336932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.337218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.337938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.338248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.340204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.347010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 374/767] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.347837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.387105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.406906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9afd5f-b514-4734-bf04-e0dd4f4895c6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.455731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.456755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.457271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.457794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.458952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.459478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.461154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.475633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 394/768] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.476773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.487884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab8515-9aba-44b1-9c5c-5f306a701775 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.524673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.525744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.526260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.526806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.527848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.528374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.530024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.537860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.543569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 375/769] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.544636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.597150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7395e07-cc5c-49ec-bbcb-3f537d2dc4f8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.635743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.636743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.637182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.637872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.638398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.639544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] RBAC: Adding query filter params (interface=public, service_id=2707fd8657ad42c8aa332eeea88fa750, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.640580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.642195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.650790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 395/770] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/endpoints?service_id=2707fd8657ad42c8aa332eeea88fa750&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.656456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.656601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de636-19eb-42aa-881c-8045446e18b9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.690065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.691103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.691488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.691841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.694454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=71160dfe2635497abc4cd1c1bffc4531, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.695980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.761174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f20368-4500-4238-b4af-bbf2e3e4314f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.780645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 376/771] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:35 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:35.781542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:35.781776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:35.828377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:35.829083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.829377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.829659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.830851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=7e4dbbeee2c847118b3fc42ae7c892f9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:35.831170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:35.833849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bba45-c935-4d81-a685-14613e1bf0de None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:35.848946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 396/772] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/role_assignments?user.id=7e4dbbeee2c847118b3fc42ae7c892f9&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:35.962039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634c8a3-0586-412d-a3db-b2d4193746b9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.962422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634c8a3-0586-412d-a3db-b2d4193746b9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.962614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634c8a3-0586-412d-a3db-b2d4193746b9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.964299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 397/773] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:35.965534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c6395-d142-4957-a2a3-b6e56371eab1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:35.970865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7291b-6b8a-4d0a-889b-f9ed5072aa5b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:35.971203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7291b-6b8a-4d0a-889b-f9ed5072aa5b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:35.971442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7291b-6b8a-4d0a-889b-f9ed5072aa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:35.991559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d3a7291b-6b8a-4d0a-889b-f9ed5072aa5b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:35.993456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d3a7291b-6b8a-4d0a-889b-f9ed5072aa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:36.022338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.023461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.023946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.024470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.025362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.027119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.034001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 377/774] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:35 2022] GET /identity/v3/services => generated 772 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:36.082528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 398/775] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:36.104882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.121451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:36.168728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8a957-bc64-4f0d-90ae-e0ce0aba2cfa None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.169196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8a957-bc64-4f0d-90ae-e0ce0aba2cfa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.169578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8a957-bc64-4f0d-90ae-e0ce0aba2cfa None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.171912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 399/776] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:36.177819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.178118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.178379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.198804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:36.200038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:36.245030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bef64-f687-46d8-a136-0d6a99103fa5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.277502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 400/777] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:36.290431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.291545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.292034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.292569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.302740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.309374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:36.319903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:36.320909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:36.321887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:36.324339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 378/778] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:36.331570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.346776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:36.411005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:36.416051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949390b7-dab2-4375-8765-f49eafd25cad None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.446313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf90a88-39c5-47ea-82f1-1d8d16ebaf5d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.453776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.454812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.455219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.455600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] PATH_INFO: `/v3/services/network` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.456409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.457897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:36.470533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:36.471668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:36.472559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 401/779] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:36.481305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.506158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.506869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.507194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.507483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.508243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:36.508583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.509959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.526322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 379/780] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:36.532508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.653540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:36.658377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2c50-d0a6-4742-a570-0b676ae72625 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.710815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631db733-f05a-47e8-93d8-f8d3bee109c3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.725368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.726047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.726337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.726620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.727338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] RBAC: Adding query filter params (name=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:36.727649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.729050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.736976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 402/781] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:36.742585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.804317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.805140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.805458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.805833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.819458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:36.845211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.846716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.852043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c11111-f856-4d98-8380-b7da2194659e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:36.890647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:36.891522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:36.891817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:36.892101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:36.892860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] RBAC: Adding query filter params (type=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:36.893173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:36.894690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:36.896945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:36.897820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:36.898708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:36.898708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:36.898708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:36.898708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:36.900023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 403/782] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:36.900445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 380/783] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:36.906489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.910560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:36.959416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:37.089634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcea2c0-99aa-4c15-b265-5957f2d54d04 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.105950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea5e78-93c7-4452-b3ab-4c0b958cea30 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.144491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.145560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.146078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.146559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.147494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:37.148014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.149693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.164572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 381/784] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:37.170603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:37.175433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.181579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.182111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.182621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.183632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] RBAC: Adding query filter params (interface=public, service_id=25f37ddb682b472aa07c2ebb0a6400d7, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:37.184153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.185785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.203665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 404/785] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:06:36 2022] GET /identity/v3/endpoints?service_id=25f37ddb682b472aa07c2ebb0a6400d7&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:37.204963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:37.266224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:37.378200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4cf34-3de3-4914-94cb-73439c8ce095 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.402882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:37.417843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.420236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d471a8-aecb-4c58-b7a0-ddbaa7405a7e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.421928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.423869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.424387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.439265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.441089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:37.471654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:37.472689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:37.473899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:37.475098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 382/786] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:37.479651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:37.502759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.503461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.503745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.504024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.504756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.506286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.520711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 405/787] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:37 2022] GET /identity/v3/services => generated 772 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:37.614688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfa4e3-f619-41cf-8545-e6d83e126582 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.622373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:37.653349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.654040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.654328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.654609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.655566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:37.655967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.657477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.680721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 383/788] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:37.689454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:37.825550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c67e0-b263-45ef-9158-8479e54fa66f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:37.863184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:37.864016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:37.864328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:37.864635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:37.875843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:37.877354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c7f17fcd-2f6e-4e0d-b6bc-254838f9d6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:37.900300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:37.902593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:37.904903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:37.904903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:37.904903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:37.904903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:37.904903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 406/789] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:37.914233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.056356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85e256-0b21-4f7f-a346-b004247a7e4b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.105088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.105767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.106060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.106348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.107094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:38.107405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:38.109653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:38.121401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c32f91-26c3-493a-a557-3af7df268905 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.121699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c32f91-26c3-493a-a557-3af7df268905 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.121976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c32f91-26c3-493a-a557-3af7df268905 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.123603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 407/790] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:38.131135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 384/791] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:38.131405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.131405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.131405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.133498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.159769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:38.161016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:38.163959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.184847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.265723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da353-77cc-4160-8c56-277a478ece6b None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.305548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.306333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.306680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.306904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.320103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:38.321614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:38.325676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 408/792] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:38.340540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:38.341498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:38.342375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 385/793] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:38.348710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.386103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.469526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda98192-2c4c-417c-8274-c66de5c25dee None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.491970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.509289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.510234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.510730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.511188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.512204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3debcdd-5a2c-48ab-a365-550f9884b9de None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.512471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:38.512962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:38.514602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9949b-394f-4a77-a74a-c2eb41ac3223 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:38.523977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.536135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 409/794] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:38.537996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa006-c5e4-4015-9f7b-fdc9b167599d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.538522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa006-c5e4-4015-9f7b-fdc9b167599d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.539088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aa006-c5e4-4015-9f7b-fdc9b167599d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.540880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 410/795] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:38.542201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa006-c5e4-4015-9f7b-fdc9b167599d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.555822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.556667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.557318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.557567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.570176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-814d2d1c-a790-461a-b493-5f946245e260 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:38.581153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:38.582131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:38.582998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 386/796] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:38.583716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f911c-e3b4-485c-a8a1-8e618b2399b2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.588997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f911c-e3b4-485c-a8a1-8e618b2399b2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.588997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f911c-e3b4-485c-a8a1-8e618b2399b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.607324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c95f911c-e3b4-485c-a8a1-8e618b2399b2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:38.609075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c95f911c-e3b4-485c-a8a1-8e618b2399b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:38.661381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aa006-c5e4-4015-9f7b-fdc9b167599d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.692616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 387/797] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:38.693889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba67a-04b6-4e29-a976-be82e2d384bf None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.694183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba67a-04b6-4e29-a976-be82e2d384bf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.694467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba67a-04b6-4e29-a976-be82e2d384bf None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.695760 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 388/798] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:38.696470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddba67a-04b6-4e29-a976-be82e2d384bf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.701813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.702742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.703026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.703306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/fe3f01883b804652b396cd4e28ab4b46/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.705341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=fe3f01883b804652b396cd4e28ab4b46, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:38.706782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791dfb7-41e9-48bc-844b-54953d32db60 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:38.772340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 411/799] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:38 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/fe3f01883b804652b396cd4e28ab4b46/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:06:38.772340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.772340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.772340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.793098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.793666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:38.794809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:38.826812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddba67a-04b6-4e29-a976-be82e2d384bf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:38.859676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:38.873862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:38.875327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:38.875618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:38.875892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:38.876699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:38.877005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:38.878674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:38.881043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 412/800] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:38.881976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:38.895935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 389/801] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:38.902767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.019596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ac905-6678-4c1d-b10f-dd28be4c214a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.030276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f8203-fd4e-4766-9f46-976eca49c578 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.056541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.057536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.057924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.058306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.069542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:39.073640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.074444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.074734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.075016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:39.078878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:39.079879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:39.080767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:39.081407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.081477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 413/802] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:39.081477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.089090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.090622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:39.119899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:39.121075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:39.122160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:39.122160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:39.122160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:39.122160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:39.122160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 390/803] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:39.122488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.141423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.196500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a26f1b-5de0-40bb-b2fc-d4b18e7cf9ec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.234210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.235218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.235608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.235991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.236794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.238265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:39.251148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:39.252137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:39.252999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 414/804] 10.222.0.47 () {56 vars in 1129 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:39.253843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.271199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79d90e-95b4-437f-9caf-4e194b5dc541 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.312173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.312902 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.313261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.313613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.314538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.314918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.316640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.325924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 391/805] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:39.326805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.369460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f4712-a59b-425b-813a-6fe74a26253f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.389183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.390768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.413135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.414134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.414630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.415154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.416104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.416690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.418282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.431769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 415/806] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:39.433152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.444853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54bdb4-5397-4060-a07f-b30e45ecd9ba None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.483247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.483746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.483856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.484145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.484905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.485221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.486624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.491860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 392/807] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:39.492895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.558792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb3d9-ecbb-4041-b04f-9eec940c98ec None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.596926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.598463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.599094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.599602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.602067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09d2e-bf0b-43f3-a587-a29072d75a94 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.611494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.613630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.617900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:39.637175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:39.638094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:39.638981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:39.638981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:39.638981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:39.638981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:39.640325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.641151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.641440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.641722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.642680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 416/808] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:39.643745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.650272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.652013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:39.670045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:39.671768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 393/809] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:39.673485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.677081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.770559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a894d8f0-5f1a-4185-9052-639f2cf46949 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.799816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa6268-4da4-49e5-ad7d-166a20171bcb None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.811123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.812119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.812799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.813318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.814801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.815336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.817073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.823045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 417/810] 10.222.0.47 () {56 vars in 1124 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:39.824440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.837398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.838078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.838366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.838646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.839386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.840039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.841249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:39.856263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 394/811] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:39.857198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:39.900373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.934122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:39.948617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9504cc-731a-46d5-8b49-0bb280ac302b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.975760 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849c6da-570c-41b8-a47c-6d74a0a96b09 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:39.989478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:39.990305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:39.990694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:39.991074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:39.992154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:39.992990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:39.994278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.017127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.017610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 418/812] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:40.018387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.018726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.019048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.019491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.020474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.022303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.127263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b7b6a-f467-4e43-8618-df29a61c6b19 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.127690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 395/813] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:06:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:40.128572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.178547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.179705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.180261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.180901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.192291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.194103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-373ad539-5e0c-43d9-8c81-e2d8d4c3a394 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:40.210962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:40.212287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:06:40.213387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:40.216045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 419/814] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:40.217681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a3d4-6ec2-46d0-a7b9-7c6a4842a3fe None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.219184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a3d4-6ec2-46d0-a7b9-7c6a4842a3fe None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.219644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a3d4-6ec2-46d0-a7b9-7c6a4842a3fe None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.221505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 420/815] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:40.222472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775a3d4-6ec2-46d0-a7b9-7c6a4842a3fe None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.226085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.268581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.283150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb4a97-66e5-48f9-9b18-88dd3b627394 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.331297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.332114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.332607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.332893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.343808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775a3d4-6ec2-46d0-a7b9-7c6a4842a3fe None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.344152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.345678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-29ade0e5-945a-4fae-87fd-ce64dcfbc91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:40.365802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:40.366768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:40.367675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:40.367675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:40.367675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:40.367675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:40.367675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 396/816] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:40.368603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e040fb-eead-42f2-940d-528dca535733 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.368603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e040fb-eead-42f2-940d-528dca535733 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.368799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e040fb-eead-42f2-940d-528dca535733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.388982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-34e040fb-eead-42f2-940d-528dca535733 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:40.391686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-34e040fb-eead-42f2-940d-528dca535733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:40.401575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.402257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.402542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.402825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.403680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:40.403990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.405527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.425362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 421/817] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:40.426214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.474655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 397/818] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:40.475619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e040fb-eead-42f2-940d-528dca535733 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.511750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.554360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfc5c2-3d62-4da7-9cab-a15a9dcfde52 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.572061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.591898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.593994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.594462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.595173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.596067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:40.596710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.599088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.607081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e040fb-eead-42f2-940d-528dca535733 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.615793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 422/819] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:40.616899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.658637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.659666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.660154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.660646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.730919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342a22-b8ff-4998-9e0e-88b79d8d085c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.752535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.755058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.756609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.770229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.771494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.772498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.773020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] PATH_INFO: `/v3/services/network` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.774102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.775684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:40.789968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:40.791119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 18:06:40.791973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:40.798953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 423/820] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:40.798953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.845615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:40.887882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 398/821] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:06:40.887882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:40.939034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b8f79-2513-4fb9-8aee-8e322b298c22 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:40.984071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:40.985186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:40.985520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:40.985808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:40.992820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:40.994267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:41.008942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:41.010067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:41.011156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:41.011716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9aeec5-57c2-4a79-91f1-dc8c133f86ec None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.013285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 424/822] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:41.014258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.052474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a6381-790e-4622-af63-f3395080d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.053148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a6381-790e-4622-af63-f3395080d019 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.053431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a6381-790e-4622-af63-f3395080d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.053716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a6381-790e-4622-af63-f3395080d019 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.054433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a6381-790e-4622-af63-f3395080d019 None admin] RBAC: Adding query filter params (name=network) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:41.054744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a6381-790e-4622-af63-f3395080d019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.056127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044a6381-790e-4622-af63-f3395080d019 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.065872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 399/823] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:41.065872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341942d-9c87-4084-ac74-90a842f2b157 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.065872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341942d-9c87-4084-ac74-90a842f2b157 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.065872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341942d-9c87-4084-ac74-90a842f2b157 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.067007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.068579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 400/824] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:41.069545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341942d-9c87-4084-ac74-90a842f2b157 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.109844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.122090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47589f85-62cd-4c12-ad62-f790937e2f37 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.160455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.161249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.161533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.161809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.172760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.174207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.176785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341942d-9c87-4084-ac74-90a842f2b157 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:41.191913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:41.192862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:41.193757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:41.193757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:41.193757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:41.193757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:41.193757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 425/825] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:41.194041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.219114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.219771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.220053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.220351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.221147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:41.221461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.222912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ab935-6a85-47d8-af7e-fedcdadb4ea5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.237048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 401/826] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:41.238158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.238460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.238742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.256678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:41.257878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:41.304938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.311042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a62df2-8c97-4e19-b6df-abd768540012 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.333497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 402/827] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:41.340519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.351795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.352437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.352729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.353016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.353743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] RBAC: Adding query filter params (type=network) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:41.354057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.355572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.362107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 426/828] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:41.362922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.373098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.460661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1aaed-870d-4495-8231-0711ae1ec16d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.473534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7d242-187f-479b-bb50-4852a55314ce None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.497916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.498520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.498806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.499226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.500196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:41.500537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.501929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.509574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.510571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.510858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.511160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/7e4dbbeee2c847118b3fc42ae7c892f9/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.518168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 403/829] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:41.518168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.532443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=7e4dbbeee2c847118b3fc42ae7c892f9, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.533893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.600040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.620016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 427/830] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:41 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/7e4dbbeee2c847118b3fc42ae7c892f9/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:06:41.620016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.625816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf649d00-dd2d-4354-97ae-acff6f088f88 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.633276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.666163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.667006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.667297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.667788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.679912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:41.689238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:41.690251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:41.691177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 404/831] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:41.692511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.769376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b835a8-8a45-446a-8c12-b1f116921833 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.812216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.813855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.814156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.814489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.815259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.815693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ad468-d0de-4ce9-8338-bf0b47aac94c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:41.816680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.857548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:41.858403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:41.858825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:41.859119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:41.866022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:41.867509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:41.883144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:41.884298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 405/832] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:41.885549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:41.894885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:41.910867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 428/833] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:06:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:41.912692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.057664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c4901-6de9-4419-a2d3-eda68b83a900 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.072713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.114236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299633cc-dbbb-4fe7-8948-8d608c378e78 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.150992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.151800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.152167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.152628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.153473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.153859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.154629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.155508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.155732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.156197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.160781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.161851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.162349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.163961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.167171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 406/834] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:42.173143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.179857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 429/835] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:42.190552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.248688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.286503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb8b8f-1e60-4d74-8276-60822858b5dc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.303842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e78a2-6279-43b7-bfea-94940e36a801 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.323761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.324894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.325366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.325830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.330601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.345604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.347078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.352045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028442cc-603a-4acc-9912-889d884a4f34 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.363006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 430/836] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:42.364412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09699b-baff-4a62-b47f-2b360e082576 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.364727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09699b-baff-4a62-b47f-2b360e082576 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.372358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09699b-baff-4a62-b47f-2b360e082576 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.372358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 431/837] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-60b5b22e-020a-456d-93c1-3ca7f7de2c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:42.372642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:42.373789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:42.375444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:42.375444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:42.375444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:42.375444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:42.375444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 407/838] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:42.377040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09699b-baff-4a62-b47f-2b360e082576 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.378661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.379025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.379375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.397171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:42.398577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:42.487935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 408/839] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:42.511940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.526165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09699b-baff-4a62-b47f-2b360e082576 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.568971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.569767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.570049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.570350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.571159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.571464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.573045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.573992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.586751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 432/840] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:42.592537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.632197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e96f0-5682-4f10-b271-5d8d63280326 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.650369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.670117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.671566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.671862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.672149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.672943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.674333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:42.688020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:42.689078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:42.689930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:42.694174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 409/841] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:42.695252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.723777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeab521-cd22-4c5f-8c5e-57baca3a6d73 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.767002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.768101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.768607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.769219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.778889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.781754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:42.799670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:42.805852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:42.806879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 433/842] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:42.807603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.813792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e225c-6329-4445-b230-e69d1b829b94 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.832080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.860370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.861913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.862361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.863525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.865146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.865614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.868038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.874832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 410/843] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:42.880344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:42.930729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3445da1-d884-4210-8a2a-99c5868b5801 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:42.943984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:42.972739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:42.973669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:42.973963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:42.974247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:42.975401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:42.975715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:42.977235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:42.997408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 434/844] 10.222.0.47 () {56 vars in 1135 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:43.002821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:43.022985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632aab1-3771-4a72-84e4-4c6a5452040a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:43.065802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:43.066635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.066931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.067217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.068263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:43.068615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:43.070294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:43.080469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 411/845] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:42 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:43.127560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca6b50-9647-4eb8-8343-c24bd3c66f33 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:43.168804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:43.169863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.170347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.171403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.211342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:43.254227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:43.255701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:43.364894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 435/846] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:06:43.370554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:43.420370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:43.480746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee34a80-e7fe-479a-a07e-75ca237dfe78 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:43.517584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:43.518394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.518685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.518969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.530099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:43.532086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:43.549869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:43.550752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:43.551836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:43.551836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:43.551836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:43.551836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:43.551836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 412/847] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:43.556781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:43.626658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:43.668280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807911b0-0d20-4476-a603-232de3b6fb70 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:43.704134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:43.704767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.705052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.705332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.706158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:43.706471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:43.707857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6f870-2162-43bf-ad21-b9f3d4990ce1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:43.720650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 436/848] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:43.726171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:43.827877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555e6b3-e0fc-48e4-9e54-5d9e1b874a76 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.828255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555e6b3-e0fc-48e4-9e54-5d9e1b874a76 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.828405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555e6b3-e0fc-48e4-9e54-5d9e1b874a76 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.829589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:43.832345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 437/849] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:43.836509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.836802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.837083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.838479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc8e3f-8113-467e-b1b8-dda5ba1ea831 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:43.854602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:43.855780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:43.876469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:43.877623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:43.878117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:43.878622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:43.890330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:43.892086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:43.907419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:43.908358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:43.909244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:43.911762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 413/850] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:43.919521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:43.936588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 438/851] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:43.960162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.026555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.063595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843768c-cd73-4833-b743-236e24da3339 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.084262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da9ba1-1054-4242-99ae-9b1ab6d694f1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.121522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.122969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.123560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.124013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.124453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.124840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.125182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.125463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.126288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:44.126804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.129150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.142863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:44.148930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 414/852] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:44.150027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:44.151230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:44.152150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:06:44.153014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:44.158370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 439/853] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:06:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:44.159277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.173114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.263886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76b56f-b368-4715-91a5-9475a85f65b9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.300167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7908e42-cfa1-4cdd-83bf-0be9355d1ae7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.302231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.303040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.303398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.303769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.305340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=fe3f01883b804652b396cd4e28ab4b46) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:44.305340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.307565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.320699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 415/854] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/role_assignments?user.id=fe3f01883b804652b396cd4e28ab4b46&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:44.328800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.349170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.349859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.350143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.350477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.351297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:44.351651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.353112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.368483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 440/855] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:44.370379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.469348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.494191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500b0b1-4f54-4563-8ab6-228b15fec770 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.537530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.538256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.538540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.538820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.550421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.551881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cd79221b-3e84-422b-950b-64d531327963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:44.570095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:44.571050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:44.571933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:44.571933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:44.571933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:44.571933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:44.573929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 416/856] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:44.577431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.625522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444ae2d-c871-40eb-a323-3f97dd7ac23d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.626580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444ae2d-c871-40eb-a323-3f97dd7ac23d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.627236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444ae2d-c871-40eb-a323-3f97dd7ac23d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.629413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 417/857] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:44.634763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66413-b959-42d4-a118-291eb32485eb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.635319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66413-b959-42d4-a118-291eb32485eb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.635840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad66413-b959-42d4-a118-291eb32485eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.653966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fad66413-b959-42d4-a118-291eb32485eb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:44.656347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fad66413-b959-42d4-a118-291eb32485eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:44.691190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceeef35-4302-4370-8a43-38de3e9ea2ee None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.728859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.729585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.729875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.730164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.736507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:44.736507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.736507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.736977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 418/858] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:44.746502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 441/859] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:44.752229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66413-b959-42d4-a118-291eb32485eb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.755240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.797650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.817185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:44.862500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad66413-b959-42d4-a118-291eb32485eb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.871784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308df51e-6d7b-468b-b407-f4784cd8d381 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:44.899328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.900229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.900454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.900735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.909769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.911194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.912895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:44.913838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:44.914137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:44.914425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:44.915095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:44.916505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:44.926313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:44.927292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:44.928226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:44.930564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:44.932501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:44.933744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 442/860] 10.222.0.47 () {56 vars in 1132 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:44.934787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 419/861] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:44.934870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:44.941549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.056182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.060383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7c684-4ef6-4d11-8cbc-36a8b3072431 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.071400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaeec99-6aaf-4c66-b2fd-e8600f0c0262 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.099137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.100495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.101550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.102042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.102522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.103583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:45.104123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.105852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.114955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.115931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.116232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.116539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.117310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:45.117623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.120274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.130300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 420/862] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:45.131516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 443/863] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:45.135074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.137791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.260874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31228a8f-f2cf-46cd-ab5c-c0a23b02408b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.269025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9eb616-9536-43ad-9cc5-4b1340f266ec None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.299880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.300519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.300807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.301246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.305733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.306447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.306677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.306954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.307658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:45.307962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.309902 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c99a36-476e-46b4-a9d7-e03ae50e9bdd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.320914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 421/864] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:45.323267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796f13-0d60-4901-b629-280888a43914 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.323881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796f13-0d60-4901-b629-280888a43914 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.324443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8796f13-0d60-4901-b629-280888a43914 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.327553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 422/865] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:45.333054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.333341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.333620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.338693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.360948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:45.362797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:45.393414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.395264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.429224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.461708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 423/866] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:45.492037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.525735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 444/867] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:06:45.532925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.592468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.613147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e8e6c-d00e-4ea7-8d94-c3427474019b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.636339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba7f2a-d4eb-4ecd-a8e9-e86b5e2771b5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.653552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.654668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.655757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.656289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.657441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.659141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.675537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.676356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.676646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.676925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.688527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.690010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:45.707965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:45.708990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:45.709900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:45.709900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:45.709900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:45.709900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:45.712373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 445/868] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:45.714588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.732808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 424/869] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:45 2022] POST /identity/v3/services => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:45.846520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25553-cfc6-4db1-96a6-26cfb9cc72b8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:45.859162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:45.892663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:45.893344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:45.893637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:45.893981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:45.894757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:45.895120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:45.896538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:45.909771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 446/870] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:45.915244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:45.960821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:46.025211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29064d31-1075-4a70-add7-e90ac9b4c2f3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:46.063004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:46.063814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.064104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.064441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.074887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:46.076528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:46.097014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:46.098343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:46.099215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 425/871] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:46.105866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:46.218941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab912-f20f-473c-8307-7d2406167042 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:46.256956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:46.258033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.258532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.259021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.260023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:46.260592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:46.262252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1630f6-fe32-4cf6-a1fd-cd3a8ae50cca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:46.276821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 447/872] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:46.283659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:46.396690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1753-5ad7-49a7-9759-60b1fc88ece4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:46.398546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:46.436889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c54633-189c-464d-b513-51431b10504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:46.438222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c54633-189c-464d-b513-51431b10504c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.438517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c54633-189c-464d-b513-51431b10504c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.438809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c54633-189c-464d-b513-51431b10504c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.439671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c54633-189c-464d-b513-51431b10504c None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=7e4dbbeee2c847118b3fc42ae7c892f9) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:46.439984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c54633-189c-464d-b513-51431b10504c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:46.446551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c54633-189c-464d-b513-51431b10504c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:46.456656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 426/873] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:46 2022] GET /identity/v3/role_assignments?user.id=7e4dbbeee2c847118b3fc42ae7c892f9&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:46.540528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee28ab-ac6a-4db3-b3f6-e0a6cf153a08 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.540863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee28ab-ac6a-4db3-b3f6-e0a6cf153a08 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.541141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee28ab-ac6a-4db3-b3f6-e0a6cf153a08 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.542817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 448/874] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:46.551053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.551502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.551892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.571385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:46.572688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:46.650462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 427/875] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:46.673817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee28ab-ac6a-4db3-b3f6-e0a6cf153a08 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:46.706750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:46.790684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee28ab-ac6a-4db3-b3f6-e0a6cf153a08 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:46.829920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:46.830740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:46.831029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:46.831309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:46.832043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:46.833497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:46.847629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:46.848922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:46.849755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 449/876] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:46.855332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:46.962918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d92be-e0ef-4326-8739-cc7382f5b9e3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:46.971883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:47.001380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:47.002128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.002419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.002702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.003492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:47.003810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:47.005251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf33ff4-0e25-415e-87a9-7b652265be2f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:47.010828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 428/877] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:47.017957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:47.121636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:47.132452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb64820-8201-41aa-abc2-71a6795135ca None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:47.168277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:47.168988 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.169279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.169561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.170280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:47.170593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:47.172618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128b011-35cc-4d54-b634-f778460b4236 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:47.177537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 450/878] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:47.398593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4a7f6-3aae-4fbd-aab8-197e3eea772a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.398882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4a7f6-3aae-4fbd-aab8-197e3eea772a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.399166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4a7f6-3aae-4fbd-aab8-197e3eea772a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.400834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 429/879] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:47.406535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a83c-6c64-4a66-b0f9-a04dfdc9bba2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.406828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a83c-6c64-4a66-b0f9-a04dfdc9bba2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.407110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3a83c-6c64-4a66-b0f9-a04dfdc9bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.424335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-31d3a83c-6c64-4a66-b0f9-a04dfdc9bba2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:47.425531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-31d3a83c-6c64-4a66-b0f9-a04dfdc9bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:47.495485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 451/880] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:47.518180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4a7f6-3aae-4fbd-aab8-197e3eea772a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:47.620718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4a7f6-3aae-4fbd-aab8-197e3eea772a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:47.655421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:47.656285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.656596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.656876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.657624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:47.659013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709f397-ebc5-487c-b72e-622b13854628 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:47.748533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 430/881] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:47 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:47.994312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d421c3-adbd-4ced-be6e-c1b2e11ff69a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:47.994670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d421c3-adbd-4ced-be6e-c1b2e11ff69a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:47.994868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d421c3-adbd-4ced-be6e-c1b2e11ff69a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:47.996597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 452/882] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:48.006402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.007361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.007952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.025675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:48.027173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:48.038856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:48.106998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 431/883] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:48.149668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d421c3-adbd-4ced-be6e-c1b2e11ff69a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.239954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:48.266766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d421c3-adbd-4ced-be6e-c1b2e11ff69a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.294506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:48.295363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.295723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.296015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.296725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:48.298112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:48.310429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:48.311722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:48.312842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 453/884] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:48.318006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.430058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca141aba-def6-4187-92ec-c81da4cd0d53 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.468061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:48.469212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.469654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.470103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.470975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:48.471437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:48.473285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:48.478922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 432/885] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:48.485644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.592132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d4742-18af-4fe1-834b-c392c1bfa98d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.628261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:48.628957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.629244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.629527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.630243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:48.630549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:48.631919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd3f91-76b6-4080-89d4-9f02975a91f7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:48.637283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 454/886] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:48.642945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.652332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:48.720505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c98c-7491-4925-b7f0-ad86a047f692 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.721182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c98c-7491-4925-b7f0-ad86a047f692 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.721681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14c98c-7491-4925-b7f0-ad86a047f692 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.723742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 455/887] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:48.728883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.729428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.729915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.748136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:48.750325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:48.756182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f054c3-d6ee-487a-a6aa-0f4842430a4c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.768655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:48.799947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:48.800965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.801258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.801539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.802633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] RBAC: Adding query filter params (interface=public, service_id=82650dca23e14604b1ee848acaf9b734, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:48.802949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:48.804516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:48.814688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 433/888] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/endpoints?service_id=82650dca23e14604b1ee848acaf9b734&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:48.820174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.832332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 456/889] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:48.852442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:48.910099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:48.947958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96f061-a4f7-4fa9-919d-7e19e0b739f0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.975971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426aa4e-8be6-4fab-902b-0137daa4479a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:48.991286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:48.992669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:48.993355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:48.993739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:48.994572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:48.996367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.005225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 434/890] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:06:49.027305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.028514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.028981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.029411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.043547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:49.053496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:49.054747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:49.055792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:49.058456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 457/891] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:06:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:49.065574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.183613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17abc2c-7480-4a52-af26-b1c81e5039d4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.224093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.224813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.225100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.225377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.226135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:49.226438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.227816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3e6c1-7ef4-4316-9d05-843abd14dfff None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.229391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:49.237810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 435/892] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:06:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:49.243321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.246341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:49.299144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8fdb-28fb-4ecb-a038-98795f18e28d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.299496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8fdb-28fb-4ecb-a038-98795f18e28d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.299706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae8fdb-28fb-4ecb-a038-98795f18e28d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.301353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 436/893] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:49.306371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.306663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.306944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.324699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:49.325988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:49.350358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13acf82c-c38e-437e-8159-1c35e9b7ff09 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.386683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.387496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.387785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.388069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.400971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.402477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.408620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 437/894] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:49.424607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:49.424779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:49.425634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:49.426561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:49.426561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:49.426561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:49.426561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:49.426561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 458/895] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:49.431825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.507243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:49.536575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59533a-a0ee-4ab5-8a95-231580d70a54 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.556378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85645909-a249-48ea-842f-4162cff3ffae None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.569989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:49.579686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.580428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.580726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.586735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.586735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.586735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.600350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.601034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.601338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.601606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.602353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:49.602663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.604035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.617572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 459/896] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:49.622816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.673731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 438/897] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:49 2022] POST /identity/v3/services => generated 241 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:49.740447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced9b15-f752-4f58-a6f0-7299ef8bf0aa None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.757538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:49.787176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.788022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.788339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.788628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.797728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.799424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-ea7d4d44-6679-4c8f-8989-8fcd45fbdb06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:49.817220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:49.818259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:49.819209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 460/898] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:49.824804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:49.935979 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353d755-e732-4e92-a920-6eee82a355db None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:49.964064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0715-b2b8-48e2-a84b-d1d4868bdafb None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.964412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0715-b2b8-48e2-a84b-d1d4868bdafb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.964660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d0715-b2b8-48e2-a84b-d1d4868bdafb None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.967187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 461/899] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:49.972159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.972481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.972761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.975808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:49.976847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:49.977347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:49.978061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:49.979271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:49.979597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:49.981154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:49.992828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:49.994341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:50.001416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 439/900] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:50.006967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.056570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.076210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 462/901] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:50.100598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.117091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db9a6e-cdbd-4834-9d05-7c67a1dd881e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.153764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.154521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.154814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.155836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.213547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfff111-7e78-4a03-9a63-16b045ee535a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.228651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.235566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.237163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.263013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.263926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.264219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.267110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.268213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.270342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.288282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:50.291628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:50.292755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:50.293794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:50.301360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 463/902] 10.222.0.47 () {56 vars in 1132 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:50.305219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.359459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 440/903] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:06:50.361314 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.439823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a7428-c775-40d0-8b17-a1aac2d66d9d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.481684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.482467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.482758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.483241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.483810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:50.484121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.485539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.490469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c11ab8-2ec6-4cf2-889f-7d271efc38da None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.495469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 464/904] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:50.496347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.531798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.535236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.536270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.536787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.537270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.551006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.553229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.557126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:50.575534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:50.576682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:50.578589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:50.578589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:50.578589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:50.578589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:50.584963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 441/905] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:50.586010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.616042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafc320-2e2f-4e17-8895-ff432fcc031e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.655190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.656564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.657304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.657747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.658865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:50.659731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.661581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.673104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 465/906] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:50.677270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.708855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5df1e9-04a1-47b6-b94f-9af9a7b5f842 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.751632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.752337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.752633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.752914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.754248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:50.754561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.756235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.776560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 442/907] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:50.778828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.804522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.813470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef0826-0b66-4d0b-a2e0-4828b7038bfd None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.843685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:50.853951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.854898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.855372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.855966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.857510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] RBAC: Adding query filter params (interface=public, service_id=28f9dd8e0f4c40588a747e554c4c0440, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:50.858084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.860919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.873239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 466/908] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/endpoints?service_id=28f9dd8e0f4c40588a747e554c4c0440&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:50.875153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:50.899276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31434c25-320f-4f12-93bb-28e91ab5d437 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:50.937633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:50.938537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:50.938909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:50.939262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:50.946453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:50.947994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:50.962951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:50.963886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:50.964817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:50.969852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 443/909] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:50.971532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.009954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7db99-3b7d-4185-839f-b57ffd75d4ce None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.058631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.059937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.060500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.061967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.063361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.065563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.073918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 467/910] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:06:51.094273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0d022-72cd-4819-ba93-c30bcb5e24ae None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.115823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:51.137272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.138046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.138234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.138515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.139444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:51.139756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.141253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da627fbf-f2cf-45cf-89b4-e8e0684eceed None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.159763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 444/911] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:51.161231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.185685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:51.195449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d7955-b71d-4c11-bdb2-8a1695152a00 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.196736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d7955-b71d-4c11-bdb2-8a1695152a00 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.197064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d7955-b71d-4c11-bdb2-8a1695152a00 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.198759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 445/912] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:51.204796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.205128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.205368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.227271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:51.228504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:51.292339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63b450-75e8-4569-8dfa-543a877f191e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.311831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 446/913] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:51.334994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.337394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.339165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.339666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.340149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.341284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] RBAC: Adding query filter params (role.id=302ba260ee0d4fcaa85ef37010149e44, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=7e4dbbeee2c847118b3fc42ae7c892f9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:51.341861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.344052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.353153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 468/914] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:06:51 2022] GET /identity/v3/role_assignments?user.id=7e4dbbeee2c847118b3fc42ae7c892f9&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=302ba260ee0d4fcaa85ef37010149e44 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:51.471173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53be23-045b-489b-984a-b2b26a8d271f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.480710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:51.511569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.512414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.512704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.512985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.513650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.515891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:51.528253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:51.529355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:51.531625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 447/915] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:51.536089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.629076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:51.645923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aef328-ce3f-4cd7-a229-c6194e06a506 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.688041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.688756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.689084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.689369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.690081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:51.690391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.691768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.697046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 469/916] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:51.703355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.809435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57042b1-2c8d-4774-ba55-0849983f9022 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.852941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:51.856479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.856479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.856479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.856479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:51.856479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:51.856835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79aaa3-0767-42b3-9f3f-00b17184cad3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:51.883153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 448/917] 10.222.0.47 () {56 vars in 1114 bytes} [Tue Jul 12 18:06:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:51.888324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:51.891507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2318d-32b6-4e6a-b0ee-19b10811c3b5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.891802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2318d-32b6-4e6a-b0ee-19b10811c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.892091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2318d-32b6-4e6a-b0ee-19b10811c3b5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.893712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 449/918] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:06:51.898649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:51.898945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:51.899230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:51.916855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:51.918063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:51.955453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:51.997130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddb1a9-88c5-4ba9-9c5b-6f6927b83124 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:51.999622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 450/919] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:52.023617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:52.039497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.040668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.041301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.041652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.042483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.044044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.128422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98578cc9-0697-4a79-a97b-f5a99e87f1c9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:52.140853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 470/920] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:06:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:52.179251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.180080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.180436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.183771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.183771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.183771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:52.197356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:52.198446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:52.199266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 451/921] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:52.204902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:52.215718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:52.326315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c42f-9f88-4978-9c2c-886ea72136ed None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:52.362279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.363012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.363310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.363595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.364378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:52.364659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.366049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.371318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 471/922] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:52.377747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:52.407738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:52.487205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3370a-3951-44bc-b418-81518b23d58e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:52.523501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.524171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.524490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.524772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.525490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:52.525858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.527250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.533111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 452/923] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:52.538912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:52.612701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:52.647662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012f1ef-eab7-4874-b16c-00673c1cea8b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:52.683550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.684154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.684473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.684751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.685559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] RBAC: Adding query filter params (interface=public, service_id=61a721ca63f34f7b9ba5be0a7d6ddf48, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:52.685871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.687428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb409ea-7d77-4681-a1bb-0cbfe05263f3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.693015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 472/924] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:06:52 2022] GET /identity/v3/endpoints?service_id=61a721ca63f34f7b9ba5be0a7d6ddf48&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:52.698579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:52.792587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:52.806147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed51a2a8-23c7-4878-8462-a4fb00b6d0f4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:52.843082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:52.843765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:52.844502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:52.844793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:52.845495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:52.847348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e0534-11d4-432a-aeec-ede53d1752d3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:52.854705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 453/925] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:52 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:06:53.242498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1215b-e641-4e15-bf59-d7a0ed04ffbc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.242907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1215b-e641-4e15-bf59-d7a0ed04ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.243086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1215b-e641-4e15-bf59-d7a0ed04ffbc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.244713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 473/926] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:53.250378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cb55-6bdd-4622-9807-f853dfddda54 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.250849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cb55-6bdd-4622-9807-f853dfddda54 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.250941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cb55-6bdd-4622-9807-f853dfddda54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.268591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c236cb55-6bdd-4622-9807-f853dfddda54 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:53.269794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c236cb55-6bdd-4622-9807-f853dfddda54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:53.341053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 454/927] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:53.365339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1215b-e641-4e15-bf59-d7a0ed04ffbc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.478641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1215b-e641-4e15-bf59-d7a0ed04ffbc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:53.501827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b30b4-8116-4ce6-a68d-9f927b24fc30 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.502118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b30b4-8116-4ce6-a68d-9f927b24fc30 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.502314 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b30b4-8116-4ce6-a68d-9f927b24fc30 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.503997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 455/928] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:53.508997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.509288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.509567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.510029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:53.517050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:53.517929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.518231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.518569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.519238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:53.520710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:53.527199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:53.528411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:53.536661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:53.537585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:53.538437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 474/929] 10.222.0.47 () {56 vars in 1132 bytes} [Tue Jul 12 18:06:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:53.546288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.623584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 456/930] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:53.647160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.655822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed7d29-ea37-4d7b-a7de-cf27b7cfeb05 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:53.662799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:53.691782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:53.692505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.692953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.693063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.693736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:53.694040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:53.695417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:53.705509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 475/931] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:53.706730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.763495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda34279-2170-4400-9fe3-890a6c008e76 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:53.768910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:53.801700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:53.802868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.803386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.803889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.809967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76914483-e8e8-4747-999a-66d199e6928f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:53.815945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:53.826046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:53.827365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:53.828428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:53.835815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 457/932] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:06:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:53.836968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.846973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:53.847644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:53.848175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:53.848506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:53.849221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:53.849526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:53.850904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:53.860389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 476/933] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:53.863072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:53.934738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:53.969578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb8190-da93-4b44-ba1c-fd8c1fa674e4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:53.997507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7949f9e-5500-4631-8ddb-904918648b37 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.018621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.019232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.019520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.019806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.020623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:54.020935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.023315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.025162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:54.035871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 458/934] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:06:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:54.040424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.040424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.040424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.040424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.040755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.040755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.042084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.122136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 477/935] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:06:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:54.157060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063ef7-f194-4479-92a0-fd79e0d269ab None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.198425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:54.205397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.212233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.212233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.212233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.219798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.221669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-7b092f5c-76d7-4ae0-bd3c-627c8a67e10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:54.241935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:54.243143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:54.244213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:54.244213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:54.244213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:54.244213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.246445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 459/936] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:54.252432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.367228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871bae4-b3f8-4a31-99b2-9383c4183b5a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.396919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97dcbe-91c2-4c3b-a5c5-75a9992730dd None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.397432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97dcbe-91c2-4c3b-a5c5-75a9992730dd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.397509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97dcbe-91c2-4c3b-a5c5-75a9992730dd None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.400147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 460/937] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:54.404112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.404443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.404810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.405828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.406816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.407281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.407734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.408668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:54.409153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.411982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.425765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:54.426987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:54.427350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 478/938] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:54.435548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.492107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:54.513567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 461/939] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:54.540320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.559363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb51f-e9e5-4b42-b5de-a43571994df5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.597562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.598841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.599226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.599587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.613052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.615445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:54.635109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:54.636051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:54.637023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:54.638809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 479/940] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:54.648340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.662803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227a251-67e3-41fc-8839-d24c70243647 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.685846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:54.718450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.719653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.720194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.720646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.721510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.724386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.726469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING keystone.server.flask.common [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 18:06:54.757693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:54.760800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d1cf-5030-42db-8459-329601e77ffb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.781688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.sql.core [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 18:06:54.781688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 18:06:54.781688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [parameters: {'id': '4e0f9076e59d416396dd2d7d9abb7f24', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 18:06:54.781688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 18:06:54.797051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.797885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.798450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.798840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.799785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:54.800223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:54.801783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4e0f9076e59d416396dd2d7d9abb7f24.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4e0f9076e59d416396dd2d7d9abb7f24. Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:54.804665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 18:06:54.805471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application yield resource Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise self.value Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application session.commit() Jul 12 18:06:54.806505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.flush() Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application compat.raise_( Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 18:06:54.807438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 18:06:54.808372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:54.809198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [parameters: {'id': '4e0f9076e59d416396dd2d7d9abb7f24', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.810075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:54.810859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4e0f9076e59d416396dd2d7d9abb7f24. Jul 12 18:06:54.812738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:54.823608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 462/941] 10.222.0.47 () {60 vars in 1140 bytes} [Tue Jul 12 18:06:54 2022] POST /identity/v3/projects => generated 262 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:54.823608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.835613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 480/942] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:54.840768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:54.952356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860ae94-7f44-4c15-b22c-2e5b1997c79a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.952847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b662979b-eb76-4bc1-b557-12d82bdb14d2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:54.991111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.991795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.992082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:54.992388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:54.997357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:54.998399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:54.999488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.000372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.003489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.004972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.009340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.010994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7a180ed8-caae-446a-9680-9eee57bc3921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:55.023129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:55.024020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 481/943] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:55.030699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6d4d-0a9e-4451-8366-e493b302975c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:55.031081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:55.031880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:55.034489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:55.035531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6d4d-0a9e-4451-8366-e493b302975c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.035531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6d4d-0a9e-4451-8366-e493b302975c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.036104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 463/944] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:55.036257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 482/945] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:55.036459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.036615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.036615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.036615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.051813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:55.052962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:55.056087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.070574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.125026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 483/946] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:55.125995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.146082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c427aea-4ef6-4ee9-9635-94d9c1439e56 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.181630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.182418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.182709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.182992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.183731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:55.184043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.185532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.200886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 464/947] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:55.201749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.245407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45647-46ba-4da4-af6c-d88012ab4306 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.283168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.283838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.284125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.284554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.285322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:55.285632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.287062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.298930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.302171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.316338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 484/948] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:55.316338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.326215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346211a-abe4-44fd-86c9-9a5252d4bef2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.376439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.377880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.378296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.378709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.379395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.381641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:55.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:55.409307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:55.410159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 465/949] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:55.415254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.438929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35d0be-4b2c-4480-9b51-a3a36c7eadf8 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.479124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.480172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.480558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.480905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.492176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.494918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:06:55.511884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:55.512842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:06:55.513880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:55.515892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 485/950] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:55.521890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.552363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b60737-a702-4b99-8731-678f43484822 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.591551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.593437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.594275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.595376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.595847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.596610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:55.596925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.598870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.604070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 466/951] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:55.610344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.633606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.642482 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1acf56-d770-4a31-bcc8-807f9c2108cd None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.683446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.684253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.684639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.685030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.685917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:55.686879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.688471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.702363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 486/952] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:55.708127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.737899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fd09b-e4b2-44a5-a598-36e4d721827f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.785394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.786583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.786876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.787315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.788215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:55.788560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.791143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.798416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 467/953] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:55.804039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:55.824644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8582ed-fb32-4e2e-95a1-421603f1f907 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.849587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.864639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.865876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.866467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.866814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/7e4dbbeee2c847118b3fc42ae7c892f9/roles/302ba260ee0d4fcaa85ef37010149e44` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.869129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=7e4dbbeee2c847118b3fc42ae7c892f9, role_id=302ba260ee0d4fcaa85ef37010149e44)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.870843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbc029-7557-4a30-a848-59e32b233ebd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:55.903847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:55.923648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d09a1d-341f-4c67-bb9b-1ef95161fd82 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:55.947887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 487/954] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:06:55 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/7e4dbbeee2c847118b3fc42ae7c892f9/roles/302ba260ee0d4fcaa85ef37010149e44 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:06:55.966535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:55.967986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:55.968366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:55.968655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:55.969641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:55.971232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91a3850-f276-408b-94c8-e8b85404a411 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:56.055539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 468/955] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:06:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:56.292564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210182ca-33d2-4bcb-b04a-914e434f29bb None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:56.292967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210182ca-33d2-4bcb-b04a-914e434f29bb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:56.293175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210182ca-33d2-4bcb-b04a-914e434f29bb None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:56.294762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 488/956] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:56.300796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:56.301094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:56.301380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:56.318771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:56.320212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:56.395283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 469/957] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:56.417536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210182ca-33d2-4bcb-b04a-914e434f29bb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:56.536360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210182ca-33d2-4bcb-b04a-914e434f29bb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:56.572399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:56.573157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:56.573452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:56.573756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] PATH_INFO: `/v3/services/compute` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:56.574441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:56.575827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:56.587990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:56.589223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:06:56.590109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:56.590888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 489/958] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:06:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:56.597338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:56.663496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:56.704981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78b48-758d-41db-b641-7ed842723174 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:56.739894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:56.740599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:56.740890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:56.741175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:56.741897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:56.742209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:56.743594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560597de-260d-4945-8483-ea9244ae9231 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:56.749484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 470/959] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:56.757712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:56.808427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:56.874192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e5d0d-0680-4002-9d9d-3c34b82e6add None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:56.911709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:56.912412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:56.912710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:56.912995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:56.913714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:56.914026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:56.915406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7838c4f-a04b-4929-bae0-832a9e74b469 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:56.920577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 490/960] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:06:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:57.610471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3ccc-1085-4dbd-a2d7-5ae3ac10dae5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:57.611472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3ccc-1085-4dbd-a2d7-5ae3ac10dae5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:57.611963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b3ccc-1085-4dbd-a2d7-5ae3ac10dae5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:57.613857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 471/961] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:57.619395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:57.620250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:57.620772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:57.638510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:57.640620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:57.712024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 491/962] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:57.736961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b3ccc-1085-4dbd-a2d7-5ae3ac10dae5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:57.839632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b3ccc-1085-4dbd-a2d7-5ae3ac10dae5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:57.866181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:57.874864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:57.875548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:57.875835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:57.876114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:57.876918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:57.878406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:57.966291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.sql.core [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 18:06:57.966291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 18:06:57.966291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [parameters: {'user_id': '472a108aaf674b7e8bc39f1ec3b78312', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 18:06:57.966291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80577) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-36026be6-8218-4e72-99bc-4ae9a186edaa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:57.976008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:57.976935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application yield resource Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application next(self.gen) Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise self.value Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application session.commit() Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 18:06:57.977920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.session.flush() Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application compat.raise_( Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 18:06:57.978850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 18:06:57.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application conn.query(q) Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application result.read() Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 18:06:57.981937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [parameters: {'user_id': '472a108aaf674b7e8bc39f1ec3b78312', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:57.984117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 18:06:57.986089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 18:06:57.988277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:57.991016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 472/963] 10.222.0.47 () {60 vars in 1131 bytes} [Tue Jul 12 18:06:57 2022] POST /identity/v3/users => generated 160 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:57.999161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.041579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.109579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6cd3ea-4821-4ed3-9ca5-21b4317a8ff6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.146125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.147275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.147795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.148278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.156228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.158071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:58.174162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:58.175179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 18:06:58.176230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:58.178861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 492/964] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:06:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:58.183444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f692-3e4b-49b1-84c3-fd1286c8846c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.183653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f692-3e4b-49b1-84c3-fd1286c8846c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.184257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f692-3e4b-49b1-84c3-fd1286c8846c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.185251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.194927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 473/965] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:58.196795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa2a92-f03c-4329-a28a-740d6da514e6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.197093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa2a92-f03c-4329-a28a-740d6da514e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.197381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa2a92-f03c-4329-a28a-740d6da514e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.216045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-04fa2a92-f03c-4329-a28a-740d6da514e6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:58.218204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-04fa2a92-f03c-4329-a28a-740d6da514e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:58.292609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.297230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042555-8627-41cb-92a6-9ffcbf38b34f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.300486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 474/966] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:06:58.301871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ec32-5e7c-4d16-bd2f-93b65bcecf56 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.306243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ec32-5e7c-4d16-bd2f-93b65bcecf56 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.306243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4ec32-5e7c-4d16-bd2f-93b65bcecf56 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.306243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 475/967] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:58.309200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126558f0-6937-4491-8de5-c0694bab085b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.312348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126558f0-6937-4491-8de5-c0694bab085b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.312348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126558f0-6937-4491-8de5-c0694bab085b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.327715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-126558f0-6937-4491-8de5-c0694bab085b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:58.329167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-126558f0-6937-4491-8de5-c0694bab085b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:58.339331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.340012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.340301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.340616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.341435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:58.341748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.344041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.378987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 493/968] 10.222.0.47 () {56 vars in 1101 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:58.379824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.424334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 476/969] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:58.439718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126558f0-6937-4491-8de5-c0694bab085b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.503889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.536246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a2dc4-d795-483c-86e2-02d6a88d6e60 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.566364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126558f0-6937-4491-8de5-c0694bab085b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.586121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.586963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.587251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.587532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.601068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:06:58.613906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:58.614891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:06:58.615821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:58.618616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.619430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.619709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.619980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.620383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 494/970] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:58.621107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.622049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.622815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:58.638139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:58.639458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:06:58.640287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 477/971] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:58.647213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.649465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.744795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bae5ec-85bc-4805-acb0-60625de33af6 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.778292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cade19-f514-44bd-b5c6-f6a027f3ed25 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.785174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.785855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.786145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.786425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.787175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:58.787494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.788902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.792119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.798104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 495/972] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:58.803676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.814575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.815295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.815587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.815869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.816676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:58.817021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.818399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.828234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 478/973] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:58.829688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.893176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:58.916539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3e2cb-9ad6-4433-8916-73a29ebd4831 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.937459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188ce7-211c-4691-80c6-62d72e193582 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:58.952458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.953536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.953820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.954113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.965462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.966901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.975011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:58.975995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.976284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.976602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.977324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:58.977830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:58.979207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0327d49-d207-4e78-86d5-a92482164105 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:58.989231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 479/974] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:06:58.989393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:58.990239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:58.991134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:06:58.991134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:06:58.991134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:06:58.991134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:58.991134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 496/975] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:06:58.991417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295ba5-222e-4699-9d62-3d9968c2a227 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:58.991417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295ba5-222e-4699-9d62-3d9968c2a227 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:58.991417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79295ba5-222e-4699-9d62-3d9968c2a227 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:58.991597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:58.991666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 480/976] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:06:58.992452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79295ba5-222e-4699-9d62-3d9968c2a227 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.034884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.090814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b11ab3-73f2-4ad8-b6b7-32691ccbec37 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.094137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79295ba5-222e-4699-9d62-3d9968c2a227 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.119078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.125405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.126069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.126359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.126641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.127463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] RBAC: Adding query filter params (interface=admin, service_id=61a721ca63f34f7b9ba5be0a7d6ddf48, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:59.127778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.128684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.129184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8395a8a-2b75-4766-ba54-1aaf9baf7936 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.129347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.129632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.129910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.130634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:59.130943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.132322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.134339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 497/977] 10.222.0.47 () {56 vars in 1249 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/endpoints?service_id=61a721ca63f34f7b9ba5be0a7d6ddf48&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:59.135765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.136062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.136371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.147117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 481/978] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:06:59.147954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.153618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:06:59.155286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:06:59.221374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 498/979] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:06:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:06:59.222224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.254355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.257408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f3979-dc05-4231-b3d5-e917734b2f1e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.292064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.292672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.292958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.293236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.293918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.295836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.301833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 482/980] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:06:59 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:06:59.302745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.330467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfba8db-6c52-4cd3-bdbd-6f8923291629 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.357595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.370690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.371526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.371814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.372094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.380904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.382411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:06:59.401863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:06:59.402915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 18:06:59.403943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:06:59.412371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 499/981] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:06:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:06:59.413458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.419277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87c5d-eb38-42f3-9079-e61ff19aa868 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.474189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.474901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.475178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.475453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.476179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.477588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.527915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cbdc8-8071-4bce-8a30-e5e0e1ff3525 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.538749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 483/982] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:06:59 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:06:59.579895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.580528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.580822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.581099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.581964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:06:59.582274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.583744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.604996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 500/983] 10.222.0.47 () {56 vars in 1149 bytes} [Tue Jul 12 18:06:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:06:59.610822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.638282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.724657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590097e-d359-4100-b428-05dc6843cad6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:06:59.759647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:06:59.760664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:06:59.761097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:06:59.761519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:06:59.832908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:06:59.834302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:06:59.859877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:06:59.945816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 484/984] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:06:59 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:06:59.947552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:06:59.990418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:00.052039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6c2f-cd62-46d6-9187-3414c2a72596 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:00.088663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:00.089376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:00.089665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:00.089940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:00.100941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:00.102442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:00.120618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:00.121561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:00.122469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:00.122469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:00.122469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:00.122469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:00.126460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 501/985] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:00.127883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:00.244620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc208868-aed9-4994-82ef-462ab76d771c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:00.270362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:00.282212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:00.282890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:00.283181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:00.283464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:00.284202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:00.284558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:00.285945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:00.302601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 485/986] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:00.304242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:00.406166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:00.415478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cd4b8-2344-47ef-aab8-006e59fff1f5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:00.463044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:00.464203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:00.464728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:00.465224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:00.473544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:00.475156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:07:00.489962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:00.491089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:00.492250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:00.494446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 502/987] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:00.500585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:00.601360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af539b6-a589-4095-8424-a6e45ad74ed1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:00.635338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:00.636012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:00.636300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:00.636608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:00.637415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:00.637728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:00.639202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99586779-79ad-4e6c-87a7-c8aca3dea7bf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:00.656408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 486/988] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:00.658359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:00.762734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259a58-2e1b-4480-bdd7-624bab36c8bf None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:00.796581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:00.797248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:00.797536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:00.797818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:00.798669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] RBAC: Adding query filter params (role.id=302ba260ee0d4fcaa85ef37010149e44, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=7e4dbbeee2c847118b3fc42ae7c892f9) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:00.798987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:00.800456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0391749-2806-4c9d-9382-f1c8e61e652b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:00.812382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 503/989] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:00 2022] GET /identity/v3/role_assignments?user.id=7e4dbbeee2c847118b3fc42ae7c892f9&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=302ba260ee0d4fcaa85ef37010149e44 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:01.392916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f10c-7ce2-4a41-b1a2-a1533495c1e4 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.393318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f10c-7ce2-4a41-b1a2-a1533495c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.393518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f10c-7ce2-4a41-b1a2-a1533495c1e4 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.395009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 487/990] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:01.400031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dab9-df10-4bbb-9ce9-364cdd6d9731 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.400544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dab9-df10-4bbb-9ce9-364cdd6d9731 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.400626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7dab9-df10-4bbb-9ce9-364cdd6d9731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.418542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-32a7dab9-df10-4bbb-9ce9-364cdd6d9731 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:01.419772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-32a7dab9-df10-4bbb-9ce9-364cdd6d9731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:01.429793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:01.497326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 504/991] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:01.521106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f10c-7ce2-4a41-b1a2-a1533495c1e4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:01.572026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:01.612010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef914a2c-f2d4-4fc5-a931-1c0454362159 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.612321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef914a2c-f2d4-4fc5-a931-1c0454362159 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.612857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef914a2c-f2d4-4fc5-a931-1c0454362159 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.614418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 505/992] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:01.620895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.621291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.621450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.632401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f10c-7ce2-4a41-b1a2-a1533495c1e4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:01.637910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:01.639698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:01.669051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:01.669833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.670129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.670407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.671062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:01.672470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:01.684505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:01.685592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 18:07:01.686578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:01.690904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 488/993] 10.222.0.47 () {56 vars in 1126 bytes} [Tue Jul 12 18:07:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:01.691727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:01.716716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 506/994] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:01.736337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:01.803276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d4555-477d-45ea-a814-24c29a1032e1 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:01.813426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:01.815287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:01.839954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01cfba5-4cb6-41c2-a5c7-6ec71051ceb5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:01.844273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:01.847812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.847812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.847812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.847812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:01.847812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:01.848526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:01.852918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 489/995] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:07:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:01.858544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:01.929817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:01.930536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:01.930823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:01.931104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:01.931774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:01.933195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:01.945500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:01.946591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:01.947451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 507/996] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:01.953516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:02.002620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735b128-89d6-4639-a9bb-b62dd28598c0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.036814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.037484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.037823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.038112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.038824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:02.039135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.040614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.050636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 490/997] 10.222.0.47 () {56 vars in 1122 bytes} [Tue Jul 12 18:07:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:02.051622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:02.063074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69165b30-3373-4bf4-beef-9a18a1cbae6e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.087167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:02.101874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.102545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.102834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.103112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.103823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:02.104134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.105541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.115263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 508/998] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:07:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:02.116106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:02.118802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:02.157909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca77fb-f4ce-4057-a00d-52c36d4a272a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.192910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.193488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.193774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.194051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.194853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.196261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.226273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293d5c7-a0c0-476f-afa4-fd0a4c49202b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.259447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.260099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.260418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.260705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.261414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:02.261729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.263114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.273753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 509/999] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:07:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:02.274570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 491/1000] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:07:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:02.277499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:02.352663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:02.391886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf61ea-3d52-4213-b7d5-7cfbee836a2e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.436631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.437473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.437764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.438046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.438856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] RBAC: Adding query filter params (interface=public, service_id=dade20ee664e41c89ad2439d13ec10fa, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:02.439174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.440585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa909a6-a33d-4cd7-9be6-907e353aee58 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.453540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 510/1001] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:07:02 2022] GET /identity/v3/endpoints?service_id=dade20ee664e41c89ad2439d13ec10fa&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:02.459289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:02.559602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:02.566958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd60fe-e67a-4e9a-8df8-82ff166d920d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:02.608225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:02.609360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.609868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.610363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.611262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:02.612888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0d7a8-fa28-4b92-bc3d-bdbf8c540880 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:02.632425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 492/1002] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:07:02 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:07:02.917575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b917-2be0-421d-8f6d-2ff26ac2a252 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.917999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b917-2be0-421d-8f6d-2ff26ac2a252 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.918191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b917-2be0-421d-8f6d-2ff26ac2a252 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.919798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 511/1003] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:02.926057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:02.926555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:02.926648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:02.943798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:02.945140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:03.016418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 493/1004] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:03.035830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2b917-2be0-421d-8f6d-2ff26ac2a252 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:03.141996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2b917-2be0-421d-8f6d-2ff26ac2a252 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:03.168019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:03.177480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:03.185229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:03.185229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:03.185229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:03.185229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:03.185229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:03.193443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:03.194429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:03.195286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:03.199623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 512/1005] 10.222.0.47 () {56 vars in 1105 bytes} [Tue Jul 12 18:07:03 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:03.200938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:03.293960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:03.304231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d801c6-7b32-423a-9a87-665a4117fa28 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:03.339204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:03.339876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:03.340163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:03.340467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:03.341177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] RBAC: Adding query filter params (name=image) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:03.341487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:03.343078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38de0ce-b393-4596-ace2-e134faaa5bdb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:03.352663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 494/1006] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:03.354230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:03.470024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc894d3-d399-41f3-b011-97abfdcb5bf5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:03.504664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:03.505333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:03.505620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:03.506020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:03.506739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:03.507047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:03.508549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2888ea-27c6-43a4-a6c3-d724876448c1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:03.516669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 513/1007] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:04.342938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a053fe3-0559-4362-8ca6-290bbc1f40a1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.343501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a053fe3-0559-4362-8ca6-290bbc1f40a1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.343501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a053fe3-0559-4362-8ca6-290bbc1f40a1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.345034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 495/1008] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:04.350213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.350515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.350797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.367616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:04.368834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:04.419405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:04.437935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 514/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:04.461791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a053fe3-0559-4362-8ca6-290bbc1f40a1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:04.566867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a053fe3-0559-4362-8ca6-290bbc1f40a1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:04.601788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:04.602507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.602792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.603069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.604270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:04.605805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bf596-f70f-49e1-86a1-ded4f5e719c1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:04.612997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 496/1010] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:04.614756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:04.718217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555662-be1f-4df3-ac56-46bd85b70ee0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:04.752998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:04.753666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.753955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.754238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.754962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:04.757399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:04.804501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c750-dcff-4029-8749-f300479cd6f2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.804798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c750-dcff-4029-8749-f300479cd6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.805083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5c750-dcff-4029-8749-f300479cd6f2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.806468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 497/1011] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:04.812207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:04.814258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:04.814258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:04.829603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:04.830801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:04.847764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:04.866396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 515/1012] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:07:04 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:07:04.906018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 498/1013] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:04.930215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:04.972394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:05.047965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc929c-e5b5-4da7-ae6d-8876b1e21830 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:05.084101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:05.084980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.085273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.085554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.086224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:05.087612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:05.101002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:05.102058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:05.102908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 516/1014] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:05.108919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:05.216542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1724896-b716-4bb3-9084-0eff7b42977d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:05.253122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:05.253802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.254091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.254373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.255098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:05.255413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:05.256842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:05.261977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 499/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:07:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:05.268588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:05.372897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:05.376939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd184532-f2ae-4151-82ee-38b327d5241f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:05.412510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:05.413194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.413486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.413765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.414487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:05.414797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:05.416445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e61479-724a-4183-9eb7-8de4ad4b1c3d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:05.427519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 517/1016] 10.222.0.47 () {56 vars in 1112 bytes} [Tue Jul 12 18:07:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:05.429245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:05.451052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:05.538323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908efc10-ba24-40ee-8a31-7130f1b64281 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:05.566824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9599-ac88-443f-8845-268d91e03aef None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.567125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9599-ac88-443f-8845-268d91e03aef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.567416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe9599-ac88-443f-8845-268d91e03aef None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.568820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 518/1017] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:07:05.573676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29e64-f57b-4826-a6dd-bf3a5f5752d1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.573979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29e64-f57b-4826-a6dd-bf3a5f5752d1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.574251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea29e64-f57b-4826-a6dd-bf3a5f5752d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.579996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:05.580692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.580986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.581267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.582016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:05.583404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:05.592098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6ea29e64-f57b-4826-a6dd-bf3a5f5752d1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:05.593589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6ea29e64-f57b-4826-a6dd-bf3a5f5752d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:05.660343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 500/1018] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:07:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:07:05.682736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 519/1019] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:05 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:05.707197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:05.783671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:05.818942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cbc49-9923-44d9-a4a2-ecd1c47f842d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:05.857064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:05.858287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:05.858576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:05.858855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:05.859603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:05.861044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df73fb7-9f0d-45be-b3d7-c51aed0f37a2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:05.922297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 501/1020] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:07:05 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:06.971158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff716e2-3ec4-40ed-b998-2d0e50c94ac4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:06.971688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff716e2-3ec4-40ed-b998-2d0e50c94ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:06.971802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff716e2-3ec4-40ed-b998-2d0e50c94ac4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:06.973531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 520/1021] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:06.978670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:06.979044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:06.979238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:06.996603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:06.997793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:07.072604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 502/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:07.099996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff716e2-3ec4-40ed-b998-2d0e50c94ac4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:07.193789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:07.205060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff716e2-3ec4-40ed-b998-2d0e50c94ac4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:07.239624 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:07.240366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:07.240656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:07.240936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:07.242111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:07.243566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a4274-c3f8-4fa1-93f3-e7cb6c7a4c12 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:07.249890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 521/1023] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:07.252388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:07.314355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:07.355705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee973c-07ff-41f0-ad52-f937986d106a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:07.390926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:07.391598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:07.391882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:07.392157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:07.392903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:07.394329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa356b9d-b0c4-4079-b922-6ffb0eb1db1d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:07.496590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 503/1024] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:07:07 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:07.798546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b939d7c-f68a-4fb0-90aa-7e59cb7d9456 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:07.798901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b939d7c-f68a-4fb0-90aa-7e59cb7d9456 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:07.799116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b939d7c-f68a-4fb0-90aa-7e59cb7d9456 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:07.800447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 522/1025] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:07.805712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2fd7-29fa-4908-9475-1d0c30204381 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:07.809075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2fd7-29fa-4908-9475-1d0c30204381 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:07.809075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2fd7-29fa-4908-9475-1d0c30204381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:07.823757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58eb2fd7-29fa-4908-9475-1d0c30204381 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:07.824990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58eb2fd7-29fa-4908-9475-1d0c30204381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:07.894938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 504/1026] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:07.918251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b939d7c-f68a-4fb0-90aa-7e59cb7d9456 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.021791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b939d7c-f68a-4fb0-90aa-7e59cb7d9456 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.035883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af77f6e-dd01-4311-8f89-3ee06cc966c2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.036445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af77f6e-dd01-4311-8f89-3ee06cc966c2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.036545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af77f6e-dd01-4311-8f89-3ee06cc966c2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.037850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 505/1027] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:07:08.042773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.043067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.043351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.060047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.060241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:08.060892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.061130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.061416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.061524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:08.073055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:08.081367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:08.082372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:08.083238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 523/1028] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:07:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:08.088295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.106463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.111957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.134715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 506/1029] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:08.157859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.196012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460455e4-07c6-4289-9986-1a30c7236020 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.233164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.233844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.234133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.234416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.235181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.235492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.236921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.250445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 524/1030] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:08.251591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.273716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6cc4d7-6e08-485d-8f0f-06b1e852f4dc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.308724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.309845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.310391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.311977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] PATH_INFO: `/v3/services/image` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.312870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.314445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:08.327593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:08.328669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:08.329690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:08.331875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 507/1031] 10.222.0.47 () {56 vars in 1105 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:08.337815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.347755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.369997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6360-eccd-4756-b1ce-9130e33ec315 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.380584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.408711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.409776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.410055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.410335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.422448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.423915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:08.443739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:08.444961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:08.445903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:08.445903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:08.445903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:08.445903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:08.446630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056f1a9-f6d8-40b6-8844-a978ea593a40 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.448909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 525/1032] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:08.449095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.487249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.488251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.488580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.488859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.489570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] RBAC: Adding query filter params (name=image) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.489875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.491250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.501523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 508/1033] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:08.503456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.559501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81896e3-2805-4818-abd5-f906aa8a9f0a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.590244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.596341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.597072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.599685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.599685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.599685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.599685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.600616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.609695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e36efa-791b-415c-beef-d3ca453bdb3d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.611509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.618316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 526/1034] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:08.619380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.647787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.648488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.648780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.649061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.649773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] RBAC: Adding query filter params (type=image) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.650266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.651984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.657877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 509/1035] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:08.663765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.735037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3fd1-4fad-4400-9b94-2f29135046a0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.773692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.774489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.774776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.775055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.780335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167000e-161b-491a-aeb4-204daaff2891 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.783061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.784561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:07:08.800036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:08.806987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 527/1036] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:08.807970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.817157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.817814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.818359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.818542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.819288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] RBAC: Adding query filter params (interface=public, service_id=e869ca34cd3b433e9ddb8afa96ac8a80, region_id=RegionOne) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.819600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.821144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.827509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 510/1037] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/endpoints?service_id=e869ca34cd3b433e9ddb8afa96ac8a80&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:08.832657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.863800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.867377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:08.919807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca90651-b16c-4118-9e3d-24bbe9acf93b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.943444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341152c-eb2a-4748-8a45-88bf1cda61a3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:08.958031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.958712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.959001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.959278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.960148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:08.960457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.961883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.979168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:08.979837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:08.985221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:08.985354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 528/1038] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:08.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:08.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:08.985450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:08.986276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:08.992579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 511/1039] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:07:09.102245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46342271-c9cf-4a63-a2aa-b544df804ebb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:09.115854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:09.141117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:09.142905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.142905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.142905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.212811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:09.214271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:09.315584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 529/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:07:09.317432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.414976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:09.421947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f521a-befe-4043-a823-2c89a17d14b3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:09.463368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:09.464172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.464500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.464783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.476353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:09.477824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-130442f2-fbc3-4217-bd61-c1e118e09ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:09.496084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:09.497053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:09.499126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:09.499126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:09.499126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:09.499126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:09.499126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 512/1041] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:09.503601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.516604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:09.605801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6baad1-5a9c-4c8c-b295-33906f41b130 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:09.620277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5e6c-3c68-4e81-9e98-ccb9d28c4360 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.620613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5e6c-3c68-4e81-9e98-ccb9d28c4360 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.620899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d5e6c-3c68-4e81-9e98-ccb9d28c4360 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.622269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 513/1042] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:09.627695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.627981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.628269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.643986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:09.644687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.644980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.645265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.645432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:09.646000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:09.646312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:09.646578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:09.647696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:09.661103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 530/1043] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:09.666478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.720357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 514/1044] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:09.741172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:09.745082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.775913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ad2a3-f343-4171-834a-4ab6a829816b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:09.814439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:09.815726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.816158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.816474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.823437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:09.824943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:09.837192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:07:09.840201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:09.841143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:09.847444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 531/1045] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:09.848246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.869775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4386a49-7bfa-42eb-b83e-3edf51e964fa None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:09.906584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:09.907671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:09.908183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:09.908654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:09.920676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:09.928206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:09.929177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:09.930024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 515/1046] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:09.935289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:09.969539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b646e5-a5df-46ac-a6a2-dbc75a9987e3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.005722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:10.011549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.012213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.012552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.012833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.013705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:10.014215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.015647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.028344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 532/1047] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:10.033177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:10.049364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af393f23-0b78-425d-8088-24f26098ff57 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.088380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.088380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.088380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.088380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.088830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:10.088830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.088830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.090937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:10.103563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 516/1048] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:10.104656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:10.143077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ed5d7-e7a9-4e32-bcc9-db2889d0f07c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.176910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.177489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.177774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.178052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.178895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] RBAC: Adding query filter params (role.id=302ba260ee0d4fcaa85ef37010149e44, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:10.179205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.180656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.186580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 533/1049] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:10 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=302ba260ee0d4fcaa85ef37010149e44 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:10.214810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a03eb-2bf1-4bd2-82ef-5e1b743f5073 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.254991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.255888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.256170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.256478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] PATH_INFO: `/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.272386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] RBAC: Authorizing `identity:get_user(user_id=5307b82b7cc543dd8e0abd1b4c70f66e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.273882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.280737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 517/1050] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:10 2022] GET /identity/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:10.282410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:10.342287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:10.389408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643003dd-a6b5-4bd6-ba54-2ddd7e8d053b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.423479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.424137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.424483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.424763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.502423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.503891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.557310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:10.603668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 534/1051] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:10.609944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:10.672171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:10.718058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acd89b-6d07-4436-8af8-e0bcdc674e45 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.754062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.754988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.755328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.755615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.756834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.758288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d7aab-40d8-4130-b1c4-26782fbc277b None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.765135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 518/1052] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:10 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:10.766666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:10.871448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6ec0-5325-4028-b3df-db63d2f6c882 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:10.915599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:10.916261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:10.916576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:10.916857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:10.917705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=553f6529721f4bdbbc3ac32c646124b9, user.id=5307b82b7cc543dd8e0abd1b4c70f66e) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:10.918022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:10.919527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d1517-586f-4e8e-bfa5-8f0de9b26723 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:10.932350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 535/1053] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:10 2022] GET /identity/v3/role_assignments?user.id=5307b82b7cc543dd8e0abd1b4c70f66e&scope.project.id=553f6529721f4bdbbc3ac32c646124b9&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:11.088224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21260a-162e-49b1-a415-e391986c4261 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.088487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21260a-162e-49b1-a415-e391986c4261 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.088757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21260a-162e-49b1-a415-e391986c4261 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.090270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 519/1054] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:11.095843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.096147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.096489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.114194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:11.115401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:11.129018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:11.185602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 536/1055] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:11.208029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21260a-162e-49b1-a415-e391986c4261 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:11.317505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21260a-162e-49b1-a415-e391986c4261 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:11.342590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:11.354868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:11.355706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.355994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.356274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.356975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:11.359054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:11.370903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:11.371998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:11.373567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 520/1056] 10.222.0.47 () {56 vars in 1105 bytes} [Tue Jul 12 18:07:11 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:11.380698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:11.488956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c4dfd-410a-4a72-afd7-26dd89dc5683 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:11.523882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:11.524574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.524860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.525134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.525838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:11.526145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:11.527500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:11.537090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 537/1057] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:11.538346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:11.644433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5ff54-6db6-472c-815d-79465c89f0b8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:11.678763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:11.679442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.679736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.680022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.680773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] RBAC: Adding query filter params (type=image) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:11.681088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:11.682472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d203201-c0b4-476a-816b-cc2767dfa651 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:11.688465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 521/1058] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:11.693591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:11.710142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:11.799101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04d402-3e27-41a0-97e8-bb60212800ed None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:11.835334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:11.835996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:11.836300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:11.836590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:11.837329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:11.838703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfae78e-f27e-4155-9191-3947eedeb2ac None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:11.905568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 538/1059] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:07:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:07:12.229515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054932-9ff0-41bc-8b9b-313b9ec75bda None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:12.229821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054932-9ff0-41bc-8b9b-313b9ec75bda None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:12.230074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66054932-9ff0-41bc-8b9b-313b9ec75bda None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:12.231472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 522/1060] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:07:12.237872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde081-662f-491c-9753-5a418390c0eb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:12.238400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde081-662f-491c-9753-5a418390c0eb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:12.238483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bde081-662f-491c-9753-5a418390c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:12.256910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-89bde081-662f-491c-9753-5a418390c0eb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:12.258111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-89bde081-662f-491c-9753-5a418390c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:12.326841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:12.344396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 539/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:12.365143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054932-9ff0-41bc-8b9b-313b9ec75bda None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:12.427814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:12.490770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66054932-9ff0-41bc-8b9b-313b9ec75bda None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:12.530382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:12.531217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:12.531509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:12.531793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:12.542828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:12.553584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:12.554600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:12.555455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:12.562054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 523/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:07:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:12.563218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde081-662f-491c-9753-5a418390c0eb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:12.679702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bde081-662f-491c-9753-5a418390c0eb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:12.719863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:12.720570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:12.720860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:12.721140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:12.721898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:12.722209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:12.723854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:12.739255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 540/1063] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:07:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:12.740542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:12.858893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcfab2-c234-40a2-9962-1297bf358ee8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:12.873427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:12.895195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:12.895998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:12.896287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:12.896596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:12.908457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:12.909954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-34bcf5cf-aedf-4f5d-b5b6-ee47436674e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:12.926878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:12.927823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:12.933183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:12.933183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:12.933183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:12.933183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:12.933183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 524/1064] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:12.935125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:12.995626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.002829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2643-cef5-4af1-a11a-973d7408ea9d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.003099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2643-cef5-4af1-a11a-973d7408ea9d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.003381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b2643-cef5-4af1-a11a-973d7408ea9d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.008923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 525/1065] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:13.010198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.010489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.010771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.028217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:13.030092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:13.050569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746070c2-abb0-4ab6-a7ae-04e15e06e535 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.087841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.088550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.088844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.089127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.089860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:13.090174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.092198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.108038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 526/1066] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:13.109899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 541/1067] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:13.113464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.131584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.215908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.224098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88dc76-e240-4bf0-bc4d-ac99c09cbecc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.242626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aa115-047a-4ed5-ae1f-ae7c802371ee None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.259557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.260388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.260679 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.260959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.264889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.269161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.270605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:07:13.285080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:13.286252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:13.287246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 527/1068] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:13.289728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.290515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.290800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.291253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.292481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.305346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:13.316156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:13.317128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:13.317987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:13.323674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 542/1069] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:13.325109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.399010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5956c9-5ab6-4282-9c69-d31bc30f86dc None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.433998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97af126-585a-4827-8056-605b46a8d40c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.434799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.436252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.436730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.437010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.437872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:13.438249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.439718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.469261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 528/1070] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:13.470278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.480721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.481458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.481774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.482029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.482953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:13.483274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.484703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.488140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.493366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 543/1071] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:13.500075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.549701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.586830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e191-b8ab-4f58-9674-d70e4070d2ec None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.613937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608dba9-60a1-471a-96cd-d132f0b1dd32 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.622003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.623357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.623656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.623939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.634248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.635707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.653486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:13.653677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:13.654637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:13.655825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:13.655825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:13.655825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:13.655825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:13.656191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.656191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.656191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] PATH_INFO: `/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.656416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 529/1072] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:13.660690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.661099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] RBAC: Authorizing `identity:get_user(user_id=5307b82b7cc543dd8e0abd1b4c70f66e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.662628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.670748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 544/1073] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:13.671957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.728180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.766714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f9e22-e2fb-4453-9b6e-18de8b083efe None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.787806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d6807-5662-4ccb-ad54-ce08f840cf5a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.796737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.805973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.806684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.806969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.807249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.807975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:13.808281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.809679 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.827038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 530/1074] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:13.827224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.827424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.827813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.827919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.828480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.829633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.837171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.837171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 545/1075] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:13.839316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:13.933673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c84c74-76df-44a2-bd4b-92e20d00c1c2 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.948955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14372e59-d181-40a6-bd26-bd6761883992 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:13.961207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:13.974126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.974976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.975259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.975609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.982276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.983811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.986790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:13.987688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:13.987977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:13.988261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9/users/5307b82b7cc543dd8e0abd1b4c70f66e/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:13.990279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] RBAC: Authorizing `identity:create_grant(project_id=553f6529721f4bdbbc3ac32c646124b9, user_id=5307b82b7cc543dd8e0abd1b4c70f66e, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:13.991711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d503a-156e-4512-a1f2-b1b47bc3348d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:07:13.999766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:14.000905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:14.001847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 531/1076] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:14.006750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:14.068711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 546/1077] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:07:13 2022] PUT /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9/users/5307b82b7cc543dd8e0abd1b4c70f66e/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:07:14.070126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3ce6-7592-4e6e-b47c-65eb833a1cd8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.070485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3ce6-7592-4e6e-b47c-65eb833a1cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.070832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c3ce6-7592-4e6e-b47c-65eb833a1cd8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.072339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 547/1078] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:14.079371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.081027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.081543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.101578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:14.102808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:14.114605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016deb35-e55f-4d1f-89fd-b6d2bd2dcca5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:14.136238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:14.155388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:14.156075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.156389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.156672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.157554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:14.158056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:14.159587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:14.178399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 532/1079] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:07:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:14.179922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:14.181756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 548/1080] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:14.205558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:14.236701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:14.298333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79d4c6-fb2d-45e6-9835-fc4c114d62e7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:14.315865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d61fb5-34b7-4e8d-bca8-7c3e18835cc4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:14.336354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:14.337250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.337535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.337814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/302ba260ee0d4fcaa85ef37010149e44` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.340087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=71160dfe2635497abc4cd1c1bffc4531, role_id=302ba260ee0d4fcaa85ef37010149e44)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:14.341602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:14.354201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:14.354852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.355138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.355442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.386842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:14.423939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:14.425424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:14.445723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 533/1081] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:07:14 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/71160dfe2635497abc4cd1c1bffc4531/roles/302ba260ee0d4fcaa85ef37010149e44 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:07:14.555534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 549/1082] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:07:14.562865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:14.680293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50bfd1f-fd80-4061-b07b-da61c0a93a64 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:14.683410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:14.717422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:14.718244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.718534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.718815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.729285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:14.730784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-46a53ea6-765f-47c8-b461-ed79c3d5e121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:14.751722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:14.752973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:14.754428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:14.754428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:14.754428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:14.754428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:14.754428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 534/1083] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:14.758792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:14.828209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:14.867925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74cfd-29d9-4e90-8573-99b2693eca48 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:14.903290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:14.905348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:14.905921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:14.906424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:14.907546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:14.907969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:14.909484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484820dc-428a-4c8a-ab9e-cd5ad2496300 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:14.923262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 550/1084] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:16.117989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5d53-9f40-49fd-a3bc-67215ef16e46 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.118559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5d53-9f40-49fd-a3bc-67215ef16e46 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.118559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca5d53-9f40-49fd-a3bc-67215ef16e46 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.119911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 535/1085] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:16.125634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.126048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.126207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.143975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:16.145335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:16.212836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 551/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:16.236810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5d53-9f40-49fd-a3bc-67215ef16e46 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.335323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca5d53-9f40-49fd-a3bc-67215ef16e46 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.369681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.370391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.370672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.370948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.381597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3d407a9f-ba74-40a9-ad83-2e94d2a99a25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:16.389804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:16.390818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:16.391669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 536/1087] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:16.397063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.453879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:16.497273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49b246-8844-4b63-b665-a7a226a24197 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.524680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05de9-6e5d-45e2-8d3b-977613303153 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.524973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05de9-6e5d-45e2-8d3b-977613303153 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.525252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05de9-6e5d-45e2-8d3b-977613303153 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.526603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 537/1088] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:16.531094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.531361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.531647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.531752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.531926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.532044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.532359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.533103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:16.533416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:16.534797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:16.543591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 552/1089] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:16.548220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:16.549201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.549432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:16.561585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:16.620662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 538/1090] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 18:07:16.644961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.651833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a3145-ff1b-41ad-843a-e7c0070339e3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.668557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:16.687650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.688571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.688862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.689147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] PATH_INFO: `/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.690363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] RBAC: Authorizing `identity:get_user(user_id=5307b82b7cc543dd8e0abd1b4c70f66e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:16.691815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:16.693509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 553/1091] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/users/5307b82b7cc543dd8e0abd1b4c70f66e => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:16.699095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.743736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f9f9c-c275-4747-93c9-538868976abe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.778248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.779198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.779670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.779921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.789966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:16.797612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9522ec-176d-40d2-a00a-21e828f51dae None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:16.798717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:16.799633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:16.800537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 539/1092] 10.222.0.47 () {56 vars in 1120 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:16.805819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:16.815512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:16.832201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.832893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.833178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.833457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.885336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:16.901328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:16.902792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:16.905882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f418b-e5bc-45bd-b7a1-04ec4acb68b5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:16.943656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:16.944349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:16.944637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:16.944918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:16.945919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:16.946359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:16.947895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:16.956740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 540/1093] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:16.962319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.005627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 554/1094] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 18:07:17.011412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.030081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.063304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8da68-1d3d-461b-a055-4097e235027c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.094784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.096654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.097442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.097725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.098250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.108924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.110371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.110490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da26b88-01b4-41f2-b504-75051fa3a89c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:17.128098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:17.129538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:17.130628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:17.130628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:17.130628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:17.130628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:17.130628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 541/1095] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:17.134908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.148035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.148926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.149213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.149495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.150677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.152120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.153959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 555/1096] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:17.159832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.235291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff239-70f3-4700-870e-be1957a16cc0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.255407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.260415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d67a9-5779-4482-8d18-a59b3524a0be None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.270176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.270845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.271134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.271414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.272147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:17.272497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.274353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.286405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 542/1097] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:17.292246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.298727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.299383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.299667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.299943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.300827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=553f6529721f4bdbbc3ac32c646124b9, user.id=5307b82b7cc543dd8e0abd1b4c70f66e) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:17.301137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.302573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.309832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 556/1098] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/role_assignments?user.id=5307b82b7cc543dd8e0abd1b4c70f66e&scope.project.id=553f6529721f4bdbbc3ac32c646124b9&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:17.396959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c63d-6831-4407-9e9c-7cf458556ebf None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.432534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.433255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.433545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.433821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.442349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.443846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:07:17.459488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:17.460698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:17.462092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 543/1099] 10.222.0.47 () {56 vars in 1093 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:17.466688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.486640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.571288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63cb61-e956-417f-9b08-34333dfd3e5c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.604642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.605316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.605605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.605889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.606740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:17.607591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.609054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.626493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 557/1100] 10.222.0.47 () {56 vars in 1133 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:17.632285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:17.674357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.732834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d56b7-80d1-47fe-a89e-87f4f5a6da2f None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:17.765707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:17.766364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:17.766652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:17.766933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:17.840666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:17.842830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:17.885699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:17.945926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 544/1101] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:17.952145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:18.047349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:18.072630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a9a8b-6e5c-4c62-81b6-905e0600d288 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:18.109545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:18.110349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:18.110639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:18.110923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:18.121511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:18.123591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:18.141498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:18.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:18.143509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:07:18.143509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:18.143509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:07:18.143509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:18.143509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 558/1102] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:07:18.148421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:18.249831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858cce-4a87-4e33-a459-90148fe113f9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:18.282463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:18.283130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:18.283424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:18.283733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:18.284619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:18.284931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:18.286342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:18.295232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:18.299147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 545/1103] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:18.304993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:18.409214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16264243-30d4-41ce-b337-7d38d141e624 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:18.436042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:18.447518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:18.448716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:18.448716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:18.449020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:18.457071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:18.458550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:07:18.474700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:18.481384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:18.484408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 559/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:07:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:18.485466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:18.582506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25967f-d32f-415c-806a-1c3cce791f82 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:18.617070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:18.617735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:18.618023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:18.618300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:18.619094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:18.619403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:18.620899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397d0b7-9e60-487c-a907-bfa771e97c02 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:18.633897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 546/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:07:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:18.640232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:18.739749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e027b50-19ac-4b5e-9e68-4890c6fae34b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:18.776498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:18.777152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:18.777436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:18.777714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:18.778552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] RBAC: Adding query filter params (role.id=302ba260ee0d4fcaa85ef37010149e44, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=71160dfe2635497abc4cd1c1bffc4531) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:18.779397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:18.780874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9b013-5952-4c9a-a9b3-2a50eada7c05 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:18.786994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 560/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:18 2022] GET /identity/v3/role_assignments?user.id=71160dfe2635497abc4cd1c1bffc4531&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=302ba260ee0d4fcaa85ef37010149e44 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:19.420404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d269f89-8b07-44ad-89fe-e4ae03b51380 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:19.421151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d269f89-8b07-44ad-89fe-e4ae03b51380 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:19.421151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d269f89-8b07-44ad-89fe-e4ae03b51380 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:19.422323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 547/1107] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:19.428209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:19.428725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:19.428819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:19.448212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:19.448212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:19.501493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:19.523516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 561/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:19.548153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d269f89-8b07-44ad-89fe-e4ae03b51380 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:19.612187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:19.653287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d269f89-8b07-44ad-89fe-e4ae03b51380 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:19.686104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:19.686833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:19.687122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:19.687405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:19.688614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:19.690060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27cac1-2b34-4526-8ce1-347259dcbd51 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:19.691685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 548/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:19.697497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:19.795802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf28e96-7df1-48fc-a82e-7b7db6bd34df None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:19.829540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:19.830207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:19.830499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:19.830781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:19.831506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:19.832989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344046a-16f9-495a-8324-b564297285d0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:19.902083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 562/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:07:19 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:21.949321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3f90e-bc39-427d-aaea-ee362abd444e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:21.949848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3f90e-bc39-427d-aaea-ee362abd444e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:21.949953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3f90e-bc39-427d-aaea-ee362abd444e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:21.951731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 549/1111] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:21.956849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:21.957266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:21.957424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:21.974712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:21.975911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:22.003266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:22.040977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 563/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:22.063720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3f90e-bc39-427d-aaea-ee362abd444e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:22.162429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3f90e-bc39-427d-aaea-ee362abd444e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:22.187181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:22.194607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:22.195453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:22.195737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:22.196012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] PATH_INFO: `/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:22.197497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] RBAC: Authorizing `identity:get_role(role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:22.198934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:22.201040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 550/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:22 2022] GET /identity/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:22.206678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:22.302671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7123c69-4039-4ce9-b790-ad3fd4786088 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:22.335917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:22.336799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:22.337081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:22.337353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] PATH_INFO: `/v3/users/4f06449c939d4b3f876165fe9ffda87e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:22.350246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] RBAC: Authorizing `identity:get_user(user_id=4f06449c939d4b3f876165fe9ffda87e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:22.351699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:22.353401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 564/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:22 2022] GET /identity/v3/users/4f06449c939d4b3f876165fe9ffda87e => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:22.359030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:22.456523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3bd2d-c587-4503-8f3a-9e740106dace None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:22.489443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:22.490084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:22.490366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:22.490639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:22.551669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:22.557905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:22.559351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:22.653931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 551/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:22.659782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:22.684658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:22.757179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36cbc7-aa97-48c7-90ac-22af3c7c8556 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:22.789214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:22.790062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:22.790343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:22.790615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:22.791774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:22.793268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d264d74-fe51-4810-b2c7-8b0b09e2e521 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:22.795499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 565/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:22 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:22.801453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:22.893457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:22.899181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9c8fa-5f0d-4447-a3c2-623bb7084b14 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:22.932091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:22.932765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:22.933051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:22.933325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:22.934159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=553f6529721f4bdbbc3ac32c646124b9, user.id=4f06449c939d4b3f876165fe9ffda87e) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:22.934468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:22.936683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acb445-6f85-4e7b-bf0c-4f4cc0ed8c7e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:22.941995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 552/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:22 2022] GET /identity/v3/role_assignments?user.id=4f06449c939d4b3f876165fe9ffda87e&scope.project.id=553f6529721f4bdbbc3ac32c646124b9&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:24.922978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39073024-14e8-4605-a9ea-101cf0c70b45 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:24.923501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39073024-14e8-4605-a9ea-101cf0c70b45 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:24.923616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39073024-14e8-4605-a9ea-101cf0c70b45 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:24.924913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 566/1118] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:24.930149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:24.930528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:24.930716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:24.947745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:24.948962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:25.013435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 553/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:25.035340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39073024-14e8-4605-a9ea-101cf0c70b45 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:25.124957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:25.134046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39073024-14e8-4605-a9ea-101cf0c70b45 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:25.166590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:25.167441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:25.167731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:25.168016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] PATH_INFO: `/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:25.170270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] RBAC: Authorizing `identity:get_role(role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:25.171655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:25.173092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 567/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:25 2022] GET /identity/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:25.178780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:25.266075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:25.275328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b04b9-e2a7-4a5d-b5ea-18ea8d3b1115 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:25.309286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:25.310379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:25.310513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:25.310738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] PATH_INFO: `/v3/users/4f06449c939d4b3f876165fe9ffda87e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:25.311922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] RBAC: Authorizing `identity:get_user(user_id=4f06449c939d4b3f876165fe9ffda87e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:25.313400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:25.315025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 554/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:25 2022] GET /identity/v3/users/4f06449c939d4b3f876165fe9ffda87e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:25.320965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:25.418916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3d1fb-400a-48ac-86eb-c00732f1711d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:25.456339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:25.457234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:25.457526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:25.457811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:25.459048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:25.460562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c9af4-b000-4b75-a432-0b878437c3f9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:25.462303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 568/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:25 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:25.469221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:25.567675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8dfb33-b293-4768-a8c8-572be1dc2633 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:25.601124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:25.601990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:25.602271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:25.602544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9/users/4f06449c939d4b3f876165fe9ffda87e/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:25.604733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] RBAC: Authorizing `identity:create_grant(project_id=553f6529721f4bdbbc3ac32c646124b9, user_id=4f06449c939d4b3f876165fe9ffda87e, role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:25.606226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3de0f-e698-44f0-a8c3-c81312bc9668 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:25.667487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 555/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:07:25 2022] PUT /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9/users/4f06449c939d4b3f876165fe9ffda87e/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:07:27.660001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36057-4553-4daf-9d42-2d1885e8d077 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:27.660545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36057-4553-4daf-9d42-2d1885e8d077 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:27.660658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a36057-4553-4daf-9d42-2d1885e8d077 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:27.661946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 569/1124] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:27.667157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:27.667565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:27.667725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:27.684793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:27.686044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:27.698741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:27.751275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 556/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:27.774096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36057-4553-4daf-9d42-2d1885e8d077 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:27.802293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:27.873544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a36057-4553-4daf-9d42-2d1885e8d077 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:27.907262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:27.908104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:27.908416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:27.908693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] PATH_INFO: `/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:27.910160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] RBAC: Authorizing `identity:get_role(role_id=1e7d33e6b3a14c0d9b112af83771b0ba)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:27.911522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:27.912891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 570/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:27 2022] GET /identity/v3/roles/1e7d33e6b3a14c0d9b112af83771b0ba => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:27.918733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:28.014635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683dd11-119c-420d-a4d0-2734f6a5e80a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:28.047649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:28.048545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:28.048835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:28.049116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] PATH_INFO: `/v3/users/4f06449c939d4b3f876165fe9ffda87e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:28.050294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] RBAC: Authorizing `identity:get_user(user_id=4f06449c939d4b3f876165fe9ffda87e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:28.051751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:28.053405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 557/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:27 2022] GET /identity/v3/users/4f06449c939d4b3f876165fe9ffda87e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:28.059088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:28.150023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:28.156477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80427a16-9e04-47ad-a0ba-6f51aad865e7 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:28.189097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:28.189737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:28.190021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:28.190295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:28.255221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:28.256685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:28.344953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:28.351435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 571/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:28.357377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:28.375649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:28.458331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ee18f-d069-4fbd-9de2-614898b11629 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:28.491586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:28.492481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:28.492770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:28.493050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] PATH_INFO: `/v3/projects/553f6529721f4bdbbc3ac32c646124b9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:28.494237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] RBAC: Authorizing `identity:get_project(project_id=553f6529721f4bdbbc3ac32c646124b9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:28.495678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081df1b-0b64-499f-af8f-773b67427f51 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:28.497959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 558/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:28 2022] GET /identity/v3/projects/553f6529721f4bdbbc3ac32c646124b9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:28.504060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:28.599438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be7f85-d438-4b18-8067-9c58619cfa3f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:28.632519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:28.633156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:28.633441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:28.633722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:28.634557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] RBAC: Adding query filter params (role.id=1e7d33e6b3a14c0d9b112af83771b0ba, scope.project.id=553f6529721f4bdbbc3ac32c646124b9, user.id=4f06449c939d4b3f876165fe9ffda87e) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:28.634870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:28.636288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b79ef-a5af-4964-8abd-1ca5f6105bf1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:28.642034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 572/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:28 2022] GET /identity/v3/role_assignments?user.id=4f06449c939d4b3f876165fe9ffda87e&scope.project.id=553f6529721f4bdbbc3ac32c646124b9&role.id=1e7d33e6b3a14c0d9b112af83771b0ba => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:30.691821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce24d0-a509-4396-bfbd-5256f744aaf2 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:30.692399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce24d0-a509-4396-bfbd-5256f744aaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:30.692399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce24d0-a509-4396-bfbd-5256f744aaf2 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:30.693790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 559/1131] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:30.699034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:30.699456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:30.699564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:30.716525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:30.717699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:30.772190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:30.783268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 573/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:30.806017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce24d0-a509-4396-bfbd-5256f744aaf2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:30.894283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:30.912059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce24d0-a509-4396-bfbd-5256f744aaf2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:31.006236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:31.006236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:31.006236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:31.006236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:31.006764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:31.006764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe1711-f0e9-4c18-ad8f-0c6b8a75a7e8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:31.006764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 560/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:31.006976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:31.060450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e757ad8-40fb-49e9-812d-c719c1da3050 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:31.093408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:31.094068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:31.094352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:31.094624 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:31.095349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:31.096845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10184c49-b656-40ef-bea2-531bd587f92b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:31.183237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 574/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:07:30 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:33.240831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f314a4a-5eed-4110-8a36-c74f3f309a73 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:33.241391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f314a4a-5eed-4110-8a36-c74f3f309a73 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:33.241391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f314a4a-5eed-4110-8a36-c74f3f309a73 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:33.242779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 561/1135] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:33.248647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:33.249093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:33.249221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:33.265535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:33.266732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:33.300934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:33.333101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 575/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:33.358812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f314a4a-5eed-4110-8a36-c74f3f309a73 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:33.457583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f314a4a-5eed-4110-8a36-c74f3f309a73 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:33.490768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:33.491597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:33.491830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:33.492104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:33.493296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:33.494728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57efe7-c17a-4aad-b4e7-eb5d0f21a7d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:33.496334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 562/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:33.502516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:33.597922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c2d65-4e0f-49b2-84cc-b480f1adca8b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:33.630547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:33.631202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:33.631489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:33.631766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:33.632501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:33.634032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de245434-d083-46c0-8b0f-658118bc2c76 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:33.723450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 576/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:07:33 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:35.770582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108483e-95c5-40fe-94ef-94d00621920e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:35.771101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108483e-95c5-40fe-94ef-94d00621920e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:35.771101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108483e-95c5-40fe-94ef-94d00621920e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:35.772454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 563/1139] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:35.777463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:35.777907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:35.778037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:35.794689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:35.795859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:35.818234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:35.861100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 577/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:35.884963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108483e-95c5-40fe-94ef-94d00621920e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:35.906781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:35.983554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108483e-95c5-40fe-94ef-94d00621920e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.017604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.018384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.018670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.018950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.028776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:36.036299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:36.037473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:36.038753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 564/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:36.043530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:36.143400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54d2c1-d309-48d1-a9b0-43889ce7ccce None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.177259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.177923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.178209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.178489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.179238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:36.179545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:36.180941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:36.189387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 578/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:36.195152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:36.272963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:36.294246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d61c-7129-4272-be4a-2e297e80da01 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.324441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.325581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.325881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.326166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] PATH_INFO: `/v3/users/8e656878ccca41edaec1ab600fa441d3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.338708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] RBAC: Authorizing `identity:get_user(user_id=8e656878ccca41edaec1ab600fa441d3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:36.340170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:36.341928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 565/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:36 2022] GET /identity/v3/users/8e656878ccca41edaec1ab600fa441d3 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:36.347948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:36.369866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:36.447807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2be423-c47b-4e44-80e7-7abff1c7f39a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.483362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.484012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.484297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.484601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.551379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:36.552804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:36.572750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:36.656523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 579/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:36.662405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:36.759117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd65f0b3-1917-4cd8-aebc-dc236e44a52e None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.791860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.792749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.793036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.793344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] PATH_INFO: `/v3/projects/b0e325d7421e4a5681445496edc8c54c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.794719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] RBAC: Authorizing `identity:get_project(project_id=b0e325d7421e4a5681445496edc8c54c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:36.796178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e0062-aedf-489a-aaf1-081f825219a0 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:36.797895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 566/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:36 2022] GET /identity/v3/projects/b0e325d7421e4a5681445496edc8c54c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:36.804164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:36.901173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375ed96-4c06-4467-b520-9a6ef03105e1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:36.920847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:36.934184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=b0e325d7421e4a5681445496edc8c54c, user.id=8e656878ccca41edaec1ab600fa441d3) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:36.939970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf703-5198-4ae9-aec9-1277f724cd70 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:36.944166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 580/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:36 2022] GET /identity/v3/role_assignments?user.id=8e656878ccca41edaec1ab600fa441d3&scope.project.id=b0e325d7421e4a5681445496edc8c54c&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:38.928162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc11d-ebb3-485a-9a9a-61effe286962 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:38.928732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc11d-ebb3-485a-9a9a-61effe286962 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:38.928732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cc11d-ebb3-485a-9a9a-61effe286962 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:38.930106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 567/1147] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:38.936087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:38.936511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:38.936773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:38.954265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:38.955481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:39.022376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 581/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:39.045353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cc11d-ebb3-485a-9a9a-61effe286962 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:39.054252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:39.147813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cc11d-ebb3-485a-9a9a-61effe286962 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:39.181447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:39.182159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:39.182445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:39.182724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:39.193700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:39.201332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:39.202404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:39.203500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 568/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:39.208795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:39.308545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f11d63-c3a2-463b-a924-07fb3fa34a58 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:39.323245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:39.342740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:39.343402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:39.343692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:39.343975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:39.344764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:39.345076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:39.346457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:39.354943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 582/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:39.361270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:39.425447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:39.462006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab3ec8-eb46-40c4-b98e-8821d8bb23e0 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:39.495004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:39.495874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:39.496159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:39.496468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] PATH_INFO: `/v3/users/8e656878ccca41edaec1ab600fa441d3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:39.498365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] RBAC: Authorizing `identity:get_user(user_id=8e656878ccca41edaec1ab600fa441d3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:39.499817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:39.501484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 569/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:39 2022] GET /identity/v3/users/8e656878ccca41edaec1ab600fa441d3 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:39.507351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:39.608322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766962-768f-48a0-8290-c8a3f85c9918 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:39.641304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:39.642172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:39.642460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:39.642742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] PATH_INFO: `/v3/projects/b0e325d7421e4a5681445496edc8c54c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:39.643938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] RBAC: Authorizing `identity:get_project(project_id=b0e325d7421e4a5681445496edc8c54c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:39.645435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fba5e-773d-4cf6-a5f0-13eb8a54064a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:39.647154 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 583/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:39 2022] GET /identity/v3/projects/b0e325d7421e4a5681445496edc8c54c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:39.653633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:39.755760 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c9ec-251e-4b10-8d7d-99904dc7210c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:39.788789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:39.789678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:39.789963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:39.790252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] PATH_INFO: `/v3/projects/b0e325d7421e4a5681445496edc8c54c/users/8e656878ccca41edaec1ab600fa441d3/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:39.792203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] RBAC: Authorizing `identity:create_grant(project_id=b0e325d7421e4a5681445496edc8c54c, user_id=8e656878ccca41edaec1ab600fa441d3, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:39.794253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfaf8b-ec18-4c45-ae5f-371a4aa7f868 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:39.796583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:39.889424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 570/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:07:39 2022] PUT /identity/v3/projects/b0e325d7421e4a5681445496edc8c54c/users/8e656878ccca41edaec1ab600fa441d3/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:07:41.880163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61bc2f-3180-4d57-a912-1a3791f60354 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:41.880712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61bc2f-3180-4d57-a912-1a3791f60354 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:41.880712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61bc2f-3180-4d57-a912-1a3791f60354 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:41.882019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 584/1154] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:41.887315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:41.887823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:41.887972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:41.904990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:41.906191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:41.971688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 571/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:41.995024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61bc2f-3180-4d57-a912-1a3791f60354 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.041592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:42.096910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e61bc2f-3180-4d57-a912-1a3791f60354 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:42.131664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:42.132400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:42.132691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:42.132974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:42.143143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:42.150793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:42.151889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:42.153758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 585/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:42.158174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.254805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6a8da-f811-4793-a757-5d9ec090bcb8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:42.277576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:42.291284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:42.291961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:42.292254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:42.292570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:42.293330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:42.294317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:42.295811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:42.304017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 572/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:42.310655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.408205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0bbb-6510-4d7b-ad9f-f9411f959482 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:42.410521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:42.443656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:42.445111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:42.446265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:42.446844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] PATH_INFO: `/v3/users/8e656878ccca41edaec1ab600fa441d3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:42.448083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] RBAC: Authorizing `identity:get_user(user_id=8e656878ccca41edaec1ab600fa441d3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:42.449561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:42.451530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 586/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:42 2022] GET /identity/v3/users/8e656878ccca41edaec1ab600fa441d3 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:42.457290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.556732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028376c-e91a-474e-932d-280c6f828c6c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:42.589661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:42.590880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:42.591168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:42.591451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:42.645353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:42.658952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:42.660378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:42.758063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 573/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:42.764097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.861845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3d316-ac4f-43e2-b109-89cbd10e8725 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:42.894821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:42.895674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:42.895957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:42.896235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] PATH_INFO: `/v3/projects/b0e325d7421e4a5681445496edc8c54c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:42.897432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] RBAC: Authorizing `identity:get_project(project_id=b0e325d7421e4a5681445496edc8c54c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:42.898870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4444a-de2b-44e0-a9b2-e547e8ca261f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:42.900589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 587/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:42 2022] GET /identity/v3/projects/b0e325d7421e4a5681445496edc8c54c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:42.907305 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:42.990743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:43.005265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571c56a-9b5f-4f2c-a536-5ca584a7c8be None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:43.041558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:43.042212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:43.042514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:43.042791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:43.044002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=b0e325d7421e4a5681445496edc8c54c, user.id=8e656878ccca41edaec1ab600fa441d3) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:43.044002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:43.045495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72e060-74ef-4a53-82dd-5daae9bf2bf7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:43.051370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 574/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:42 2022] GET /identity/v3/role_assignments?user.id=8e656878ccca41edaec1ab600fa441d3&scope.project.id=b0e325d7421e4a5681445496edc8c54c&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:45.151005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e68bb-1344-4b6f-bb10-dcfcf8c22357 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:45.151545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e68bb-1344-4b6f-bb10-dcfcf8c22357 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:45.151545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e68bb-1344-4b6f-bb10-dcfcf8c22357 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:45.153593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 588/1162] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:45.158900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:45.159318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:45.159470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:45.176425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:45.177628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:45.243939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 575/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:45.266811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e68bb-1344-4b6f-bb10-dcfcf8c22357 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:45.308803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:45.368465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e68bb-1344-4b6f-bb10-dcfcf8c22357 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:45.401667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:45.402236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:45.402521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:45.403346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:45.469098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:45.470571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:45.510157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:45.566762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 589/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:45.572872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:45.651447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:45.670155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c882fc4-8ddb-45d5-a7bf-d663907bd8a7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:45.705136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:45.705953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:45.706241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:45.706527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:45.713356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:45.714817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9758f48c-6f0d-43a9-afd7-2b358c0ff1ad None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:45.733273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:45.743263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:07:45.744293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 576/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:07:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:45.745318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:45.838906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d6bda-c411-42e8-88d4-09cfec643f84 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:45.873311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:45.873968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:45.874256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:45.874539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:45.875262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:45.875571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:45.876977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fcb3e-f79f-4eeb-9fef-d54c7d4f2fbe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:45.883907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 590/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Tue Jul 12 18:07:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:47.891784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5983-e57e-4ac1-8b28-a331b3316b7f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:47.892353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5983-e57e-4ac1-8b28-a331b3316b7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:47.892353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5983-e57e-4ac1-8b28-a331b3316b7f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:47.893835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 577/1167] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:47.898948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c8138-26bb-4286-a50d-404755867b0c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:47.899364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c8138-26bb-4286-a50d-404755867b0c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:47.899519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c8138-26bb-4286-a50d-404755867b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:47.905742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:47.916164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b16c8138-26bb-4286-a50d-404755867b0c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:47.917376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b16c8138-26bb-4286-a50d-404755867b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:47.983664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 591/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:48.006363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca5983-e57e-4ac1-8b28-a331b3316b7f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:48.107634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca5983-e57e-4ac1-8b28-a331b3316b7f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:48.140606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:48.141264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:48.141550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:48.141885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:48.142588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:48.143962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb436cd-a219-459c-b92b-6b2c55a558f4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:48.204100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:48.210312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 578/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Tue Jul 12 18:07:48 2022] POST /identity/v3/domains => generated 267 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:50.264934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d80ae-77da-4828-a813-1035a4f09a5d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:50.265470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d80ae-77da-4828-a813-1035a4f09a5d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:50.265470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d80ae-77da-4828-a813-1035a4f09a5d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:50.266854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 592/1170] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:50.272025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:50.272462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:50.272627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:50.290017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:50.291231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:50.358212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 579/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:50.382996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d80ae-77da-4828-a813-1035a4f09a5d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:50.483576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d80ae-77da-4828-a813-1035a4f09a5d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:50.516470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:50.517260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:50.517546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:50.517823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] PATH_INFO: `/v3/domains/dddfa6d7ec2d4e6e81fc5e0ed5e1e79c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:50.519703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:50.532870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dddfa6d7ec2d4e6e81fc5e0ed5e1e79c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:50.543743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6289b-9967-46b8-8429-6ecefc19dcf4 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:50.543743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 593/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Tue Jul 12 18:07:50 2022] GET /identity/v3/domains/dddfa6d7ec2d4e6e81fc5e0ed5e1e79c => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:50.544121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:50.637492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c2921-187d-45ce-a5dd-05c35d506e2e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:50.664155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:50.670041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:50.674941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:50.674941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:50.674941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:50.674941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:50.674941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb493c-37ce-4316-a7c5-69f521adb6e1 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:50.766218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 580/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:07:50 2022] POST /identity/v3/projects => generated 361 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:52.836973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420944cd-8a09-4f9d-bfb1-e8d6ae6261f1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:52.837483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420944cd-8a09-4f9d-bfb1-e8d6ae6261f1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:52.837483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420944cd-8a09-4f9d-bfb1-e8d6ae6261f1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:52.838840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 594/1174] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:52.844804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:52.845218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:52.845364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:52.862469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:52.863653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:52.928619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 581/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:52.953708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420944cd-8a09-4f9d-bfb1-e8d6ae6261f1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:53.053666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420944cd-8a09-4f9d-bfb1-e8d6ae6261f1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:53.086432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:53.087341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:53.087578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:53.087855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] PATH_INFO: `/v3/domains/dddfa6d7ec2d4e6e81fc5e0ed5e1e79c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:53.089055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dddfa6d7ec2d4e6e81fc5e0ed5e1e79c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:53.091109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da590dd2-39a9-4e1c-8e88-0e54a1d56842 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:53.092753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 595/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Tue Jul 12 18:07:52 2022] GET /identity/v3/domains/dddfa6d7ec2d4e6e81fc5e0ed5e1e79c => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:53.098719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:53.193263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b617c1-4d58-4a43-a84a-cda28c78a7fe None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:53.206047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:53.226671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:53.227340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:53.227625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:53.227903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:53.228638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:53.230079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cced4-0a5f-4e2f-9e1e-7012fb2f1e62 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:53.306644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 582/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:07:53 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:07:55.379128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f37b6-0530-41f3-8d95-53dc8c479809 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:55.379671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f37b6-0530-41f3-8d95-53dc8c479809 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:55.379671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f37b6-0530-41f3-8d95-53dc8c479809 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:55.381176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 596/1178] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:55.386226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:55.386659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:55.386802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:55.403904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:55.405142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:55.470980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 583/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:55.494507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f37b6-0530-41f3-8d95-53dc8c479809 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:55.554909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:55.594567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f37b6-0530-41f3-8d95-53dc8c479809 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:55.627034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:55.627802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:55.628085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:55.628387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:55.638691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:55.646227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:55.647328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:55.648421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 597/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:55.654079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:55.746884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:55.750930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182a3e8-8ba0-4442-aa3d-a656d850b5af None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:55.786309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:55.786986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:55.787481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:55.787788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:55.788939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:55.789259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:55.790654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:55.798847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 584/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:55.804906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:55.904412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec3efc-498f-40df-9fdc-93c2035ecabb None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:55.926356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:55.937634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:55.938512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:55.938803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:55.939089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] PATH_INFO: `/v3/users/591097f1280c49c38b4d73bac5231044` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:55.953201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] RBAC: Authorizing `identity:get_user(user_id=591097f1280c49c38b4d73bac5231044)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:55.954655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:55.956379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 598/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:55 2022] GET /identity/v3/users/591097f1280c49c38b4d73bac5231044 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:55.962237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:56.059415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3afc7-c619-46e5-97cc-d5245c0e291d None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:56.092216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:56.092912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:56.093202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:56.093483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:56.121654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:56.163874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:56.165288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:56.261173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 585/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:07:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:07:56.267081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:56.364104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ee60-d1c7-4664-bfbc-482b490265f0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:56.396766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:56.397621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:56.397902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:56.398176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] PATH_INFO: `/v3/projects/b84ce54e99ad4551874688331aefd7bf` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:56.399343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] RBAC: Authorizing `identity:get_project(project_id=b84ce54e99ad4551874688331aefd7bf)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:56.400808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01659f65-36b5-4e81-82ed-ab85425b6380 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:56.403093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 599/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:56 2022] GET /identity/v3/projects/b84ce54e99ad4551874688331aefd7bf => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:56.409232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:56.467114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:56.505018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0999388-3e3e-474f-9151-73f59ca9e0e5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:56.538027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:56.538676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:56.538960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:56.539244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:56.540075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=b84ce54e99ad4551874688331aefd7bf, user.id=591097f1280c49c38b4d73bac5231044) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:56.540418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:56.541851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30886a90-c1c6-40dc-90a0-2327a015c415 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:56.547250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 586/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:07:56 2022] GET /identity/v3/role_assignments?user.id=591097f1280c49c38b4d73bac5231044&scope.project.id=b84ce54e99ad4551874688331aefd7bf&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:58.537041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891390c-ed03-4dc0-a3c3-b15b164b97aa None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:58.537568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891390c-ed03-4dc0-a3c3-b15b164b97aa None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:58.537685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891390c-ed03-4dc0-a3c3-b15b164b97aa None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:58.538958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 600/1186] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:07:58.544159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:58.544615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:58.544767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:58.562029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:07:58.563243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:07:58.632029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 587/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:07:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:07:58.654731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891390c-ed03-4dc0-a3c3-b15b164b97aa None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:58.706099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:58.758240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891390c-ed03-4dc0-a3c3-b15b164b97aa None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:58.791021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:58.791814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:58.792101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:58.792408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:58.803327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:07:58.811605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:07:58.812759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:07:58.813802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 601/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:07:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:07:58.819179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:58.873076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:58.917519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1407d5f-b939-4bba-ba63-c2039944cc9d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:58.950066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:58.950793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:58.951086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:58.951373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:58.952137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:07:58.952483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:58.953873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:58.962666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 588/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:07:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:58.969175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:59.071663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b904b-8543-46a2-88d2-3d770f9760e2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:59.081291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:59.106765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:59.107917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:59.108210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:59.108527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] PATH_INFO: `/v3/users/591097f1280c49c38b4d73bac5231044` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:59.109723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] RBAC: Authorizing `identity:get_user(user_id=591097f1280c49c38b4d73bac5231044)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:59.111164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:59.112839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 602/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:07:58 2022] GET /identity/v3/users/591097f1280c49c38b4d73bac5231044 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:07:59.118973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:59.216582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3523db6-e8b8-44de-ab61-f51a08508802 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:59.233091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:07:59.250669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:59.251536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:59.251825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:59.252107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] PATH_INFO: `/v3/projects/b84ce54e99ad4551874688331aefd7bf` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:59.253325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] RBAC: Authorizing `identity:get_project(project_id=b84ce54e99ad4551874688331aefd7bf)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:59.254771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72a4d-f1b8-4299-b1c6-134203a4a17d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:59.256646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 589/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:07:59 2022] GET /identity/v3/projects/b84ce54e99ad4551874688331aefd7bf => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:07:59.263230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:07:59.360165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39d495-c40f-4d49-9f18-57a3d651a758 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:07:59.392476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:07:59.393351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:07:59.393636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:07:59.393914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] PATH_INFO: `/v3/projects/b84ce54e99ad4551874688331aefd7bf/users/591097f1280c49c38b4d73bac5231044/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:07:59.395840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b84ce54e99ad4551874688331aefd7bf, user_id=591097f1280c49c38b4d73bac5231044, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:07:59.397302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6493cd-4618-49dd-b247-c5f69e0b14a5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:07:59.450239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 603/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:07:59 2022] PUT /identity/v3/projects/b84ce54e99ad4551874688331aefd7bf/users/591097f1280c49c38b4d73bac5231044/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:08:01.463534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126edd6-5366-4e27-b6a2-63b8589c0af3 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:01.463990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126edd6-5366-4e27-b6a2-63b8589c0af3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:01.464103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126edd6-5366-4e27-b6a2-63b8589c0af3 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:01.466187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 590/1193] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:08:01.471471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:01.471868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:01.472039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:01.490055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:08:01.491261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:08:01.519009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:01.556664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 604/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:08:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:08:01.581063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126edd6-5366-4e27-b6a2-63b8589c0af3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:01.687871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126edd6-5366-4e27-b6a2-63b8589c0af3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:01.721528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:01.722242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:01.722774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:01.723050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:01.733298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:08:01.740837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:08:01.741994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:08:01.743186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 591/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:08:01.748760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:01.843538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa5ba6-ed39-4fd5-805c-08a487156e70 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:01.877054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:01.877714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:01.877999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:01.878276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:01.879023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:08:01.879331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:08:01.880729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:08:01.888675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 605/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:08:01.894490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:01.939958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:01.993219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a2229-97d6-46d0-a172-361753a379bd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:02.024852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:02.025710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:02.025992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:02.026268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] PATH_INFO: `/v3/users/591097f1280c49c38b4d73bac5231044` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:02.027965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] RBAC: Authorizing `identity:get_user(user_id=591097f1280c49c38b4d73bac5231044)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:08:02.029435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:08:02.031034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 592/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Tue Jul 12 18:08:01 2022] GET /identity/v3/users/591097f1280c49c38b4d73bac5231044 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:08:02.036801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:02.065084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:02.135815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e006f-bca7-4ae6-b36b-ed3b3edbf97a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:02.171717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:02.172400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:02.172693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:02.172976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:02.239991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:08:02.241499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:08:02.267615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:02.341377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 606/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:08:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:08:02.347018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:02.443221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f9e0a-9bbc-48b9-b232-3a6c377272a4 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:02.457058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:02.478138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:02.479004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:02.479292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:02.479576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] PATH_INFO: `/v3/projects/b84ce54e99ad4551874688331aefd7bf` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:02.480791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] RBAC: Authorizing `identity:get_project(project_id=b84ce54e99ad4551874688331aefd7bf)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:08:02.482281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1292fb-d227-4f91-a121-34e3549035d3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:08:02.484012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 593/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Tue Jul 12 18:08:02 2022] GET /identity/v3/projects/b84ce54e99ad4551874688331aefd7bf => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:08:02.490109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:08:02.586267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b5032-3b80-462a-b7d7-67c38981b1e2 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:08:02.611864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:02.619534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=b84ce54e99ad4551874688331aefd7bf, user.id=591097f1280c49c38b4d73bac5231044) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:08:02.624431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0bd08f-0b8a-4255-91ce-f4919d050def None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:08:02.629889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 607/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:08:02 2022] GET /identity/v3/role_assignments?user.id=591097f1280c49c38b4d73bac5231044&scope.project.id=b84ce54e99ad4551874688331aefd7bf&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:08:12.494590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=10 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:08:12.643393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:14.482384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435006ea-87bd-4105-b88f-226ed256088b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:14.482992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435006ea-87bd-4105-b88f-226ed256088b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:14.482992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435006ea-87bd-4105-b88f-226ed256088b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:14.484360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 594/1201] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:14.489662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:14.490114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:14.490219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:14.508641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:14.510642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:14.576834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 608/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:14.601695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435006ea-87bd-4105-b88f-226ed256088b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:14.707141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435006ea-87bd-4105-b88f-226ed256088b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:14.741975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:14.742771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:14.743055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:14.743335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:14.754935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:14.756582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:14.776175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:14.777445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:14.778652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:14.778652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:14.778652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:14.778652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:14.778652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 595/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:14.783529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:14.881924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672244-cea1-41b9-b626-34e839917b6c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:14.912643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:14.918990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:14.919661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:14.919951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:14.920234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:14.920997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:14.921309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:14.922713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb8876-3abe-4bf8-a980-4aed65ea46d9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:14.935072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 609/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:14.941077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:15.000838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:15.051987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce1aa8-0af3-4bf2-9f0a-42e468c9561c None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:15.087436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:15.089877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:15.089877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:15.089877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:15.089877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:15.091063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb315-438b-4a33-814f-0f8327228cdc None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:15.173346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 596/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Tue Jul 12 18:09:14 2022] POST /identity/v3/users => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:09:17.254093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e3b-a3aa-4379-adf2-ae82588bad83 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:17.254654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e3b-a3aa-4379-adf2-ae82588bad83 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:17.254654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b87e3b-a3aa-4379-adf2-ae82588bad83 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:17.256065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 610/1206] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:17.261672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:17.262253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:17.262253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:17.280879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:17.288513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:17.351916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 597/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:17.375807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87e3b-a3aa-4379-adf2-ae82588bad83 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:17.488365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b87e3b-a3aa-4379-adf2-ae82588bad83 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:17.525926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:17.526643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:17.526930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:17.527212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:17.537985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:17.545967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:17.547153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:17.548403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 611/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:17.553357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:17.597084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:17.656212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e77d5-7f91-425f-9300-86c34748c097 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:17.691078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:17.691755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:17.692043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:17.692348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:17.693107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:17.693419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:17.694834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:17.703583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 598/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:09:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:17.709318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:17.762662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:17.812719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda44ae-4246-4e8d-8f3a-4e7b714d7927 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:17.850433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:17.851255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:17.851773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:17.852066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:17.868337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:17.870338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:17.888538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:17.890959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:17.891869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:17.891869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:17.891869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:17.891869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:17.891869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 612/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:17.895656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:17.995302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:17.998221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45d86-ae73-4265-bd51-cd8093665e40 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:18.032163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:18.032862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:18.033172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:18.033575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:18.034211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:18.034521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:18.035911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:18.047416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 599/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:18.053074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:18.158992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02211437-9d96-4990-9e75-da21b47f7f2c None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:18.160602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:18.196118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:18.196985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:18.197272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:18.197549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:18.205012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:18.206465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:18.221715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:18.227199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:18.228938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 613/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:18.229773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:18.335909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5054d80-1d31-4ec4-8812-2a7dd61dee15 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:18.370544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:18.371233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:18.371513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:18.371791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:18.372687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:18.373006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:18.374495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:18.396932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 600/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:18.403682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:18.512095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316eccf-4a21-40b3-8cf9-f39c036aee9e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:18.547395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:18.548189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:18.548508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:18.548792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:18.580131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:18.617398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:18.619378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:18.731612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 614/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:09:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:09:18.738009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:18.758484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:18.840372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d459d-963a-418b-b69c-9bdaba8cccf9 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:18.875338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:18.876130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:18.876447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:18.876728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:18.887327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:18.888852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:18.906744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:18.907681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:18.908747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:18.908747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:18.908747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:18.908747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:18.908747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 601/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:18.914627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:18.975944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:19.015775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5edd0b-91de-4bce-b4a6-3f18f28bf04b None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:19.054567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:19.055235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:19.055522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:19.055804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:19.056943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:19.056943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:19.058318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:19.070693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 615/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:19.077394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:19.150700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:19.184573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbc78d-a4cf-47ed-9a56-7576e7cd68d7 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:19.218611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:19.219410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:19.219694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:19.219973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:19.226951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:19.229459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:19.243287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:19.245209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:19.246500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 602/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:19.251323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:19.353588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892834d7-390b-41b7-b05e-a2a426e8d74d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:19.376649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:19.388644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:19.389315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:19.390096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:19.390096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:19.390801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:19.391114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:19.392590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46357eb1-2145-4160-ab43-b269d5dba0b9 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:19.404618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 616/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:19.410783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:19.518428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac92ee-46b2-432d-83fa-47883d598f44 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:19.547058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:19.553146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:19.553813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:19.554097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:19.554375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:19.555223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=096d7e0380e84c97a9b72b4203a19abd) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:19.555542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:19.557800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2150d-c43d-4731-9140-10d345a9b51d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:19.563707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 603/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:09:19 2022] GET /identity/v3/role_assignments?user.id=096d7e0380e84c97a9b72b4203a19abd&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:21.610671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be52bc-94fa-403f-8862-96151438eee6 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:21.611235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be52bc-94fa-403f-8862-96151438eee6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:21.611235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be52bc-94fa-403f-8862-96151438eee6 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:21.612640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 617/1220] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:21.618186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:21.621527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:21.621527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:21.637880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:21.639104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:21.709367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 604/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:21.733919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be52bc-94fa-403f-8862-96151438eee6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:21.845206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be52bc-94fa-403f-8862-96151438eee6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:21.881148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:21.881945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:21.882228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:21.882505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:21.893750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:21.901532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:21.904636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:21.905512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 618/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:09:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:21.908988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.011620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56a81b-a75b-49f6-876a-a83c3430381f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.036989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:22.048667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.049370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.050933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.050933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.050933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] RBAC: Adding query filter params (name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:22.051287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.052451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.061731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 605/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:09:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:22.069181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.119684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:22.178829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53868-48c4-4333-acca-216dae6aac9a None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.213722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.214520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.214809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.215090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.229000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.230490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:22.247980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:22.249115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:22.250013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:22.250013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:22.250013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:22.250013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:22.250013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 619/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:22.255090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.362652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacc604-55d5-4387-a527-af92b64e4201 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.396838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.397497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.397785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.398062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.398793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:22.399100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.401240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.413344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 606/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:22.419797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.519503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:22.527606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814072f-e0ac-467d-b3ce-c868939ca165 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.562825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.563635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.563926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.564208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.572064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.573577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:22.589067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:22.591042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:22.592240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 620/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:22.596221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.643659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:22.700315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa6d4a-66e4-4888-803e-7963f435a954 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.736095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.736863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.737153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.737429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.738289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:22.738599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.740092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.758117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 607/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:22.764184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:22.870112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094446eb-69e4-4574-b840-73fc977f1530 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:22.905515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:22.906238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:22.906525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:22.906808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:22.917536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:22.919026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:22.938742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:22.939932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:22.946412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:22.946412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:22.946412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:22.946412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:22.946412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 621/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:22.947693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:23.050017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310f0ca-328f-4473-afc6-b68843e519a8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:23.052361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:23.089645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:23.090323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:23.090614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:23.090919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:23.091657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:23.091965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:23.093390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:23.106074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 608/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:23.112109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:23.123581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:23.222019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6403-fe24-48ef-99e8-38ea7cbbfce1 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:23.257132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:23.257942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:23.258231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:23.258514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:23.266093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:23.267555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:23.281313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:23.282351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:23.283474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 622/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:23.288570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:23.393287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb8bb4c-acc2-4c37-a3bb-ac86004d8416 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:23.434655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:23.435570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:23.436001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:23.436465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:23.437592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:23.437909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:23.439484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd114b-16e6-4d9f-b6a5-6f6c70da546d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:23.457511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 609/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:09:23.463995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:23.535497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:23.570773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44e67e-6aba-4c72-9335-6f07b6568691 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:23.605023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:23.605944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:23.606225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:23.606498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/096d7e0380e84c97a9b72b4203a19abd/roles/85e25458ed46458b8202ba24c3fe7f7e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:23.623341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=096d7e0380e84c97a9b72b4203a19abd, role_id=85e25458ed46458b8202ba24c3fe7f7e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:23.625234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e38578-2c63-403e-bee8-3c723b567395 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:23.692106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 623/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:09:23 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/096d7e0380e84c97a9b72b4203a19abd/roles/85e25458ed46458b8202ba24c3fe7f7e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:09:25.737081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cb89-b458-4625-85a4-77d92615192d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:25.737629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cb89-b458-4625-85a4-77d92615192d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:25.737629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25cb89-b458-4625-85a4-77d92615192d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:25.738968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 610/1233] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:25.745189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:25.745669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:25.745767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:25.764239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:25.765486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:25.835272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 624/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:25.859950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25cb89-b458-4625-85a4-77d92615192d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:25.887261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:25.968646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25cb89-b458-4625-85a4-77d92615192d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.004445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.005162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.005453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.005736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.017018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:26.025975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:26.027118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:26.028145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 611/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Tue Jul 12 18:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:26.034287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.080973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:26.143489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1fdfb-49b0-4433-aa91-4a881bfc5c74 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.177986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.178657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.178947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.179232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.179988 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] RBAC: Adding query filter params (name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:26.180297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:26.181958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:26.189779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 625/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:09:26.195662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.281289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:26.296793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95eb88b-2bbf-454d-b5de-cbbb3c6a5e25 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.330103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.330891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.331176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.331452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.342074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:26.343533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:26.362343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:26.363431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:26.364986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:26.364986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:26.364986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:26.364986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:26.364986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 612/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:26.369534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.469685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac75ee-cb9f-419e-8d34-56f5c2870341 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.478857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:26.502721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.503382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.503672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.503953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.504734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:26.505043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:26.506435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:26.518979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 626/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:09:26.525242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.631733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380d31-7c39-4668-a361-706883f54ecd None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.666532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.667331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.667613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.667890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.670661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:26.675319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:26.676817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:26.692190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:26.704229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:26.705464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 613/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:09:26.706207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.805392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6d781-6033-4b21-aacf-dd1e6f90ae0e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:26.845019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:26.845704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:26.845993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:26.846276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:26.847385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:26.847696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:26.849158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:26.866976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 627/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:26.873362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:26.975828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a47f48-79f9-416e-8f9c-227985c41d91 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.010767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.011429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.011734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.011998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.076327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:27.083580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:27.085158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:27.188645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 614/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:09:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:09:27.194457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:27.222519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:27.299020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c31504-3595-4a97-b6ca-132f9e88591f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.334118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.334916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.335206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.335491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.346095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:27.347803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:27.366234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:27.367370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:27.378530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:27.378530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:27.378530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:27.378530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:27.378530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 628/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:27.378930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:27.472648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:27.479240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c6147-3972-4d7c-ac02-c20797121500 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.513322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.513989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.514277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.514559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.515292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:27.515603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:27.517130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:27.528987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 615/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:27.534981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:27.615016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:27.635966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a671-8d75-414b-aa8f-a252fcfccd90 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.671090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.671904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.672188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.672509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.679339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:27.681326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:27.694974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:27.696089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:27.697422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 629/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:27.702366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:27.803616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7670d3-0962-4699-8eab-8fca8a559412 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.838020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.838692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.838979 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.839260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.840058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:27.840400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:27.841867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ecdf16-e3bc-4e6f-82bf-086ecabd8716 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:27.854170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 616/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:27.860258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:27.962498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894150f3-7736-4c08-8a82-978dac67452d None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:27.997461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:27.998110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:27.998416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:27.998826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:27.999562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] RBAC: Adding query filter params (role.id=85e25458ed46458b8202ba24c3fe7f7e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=096d7e0380e84c97a9b72b4203a19abd) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:27.999874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:28.001360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03597609-e965-43b0-a90c-7f6f578d58e3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:28.007295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 630/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:09:27 2022] GET /identity/v3/role_assignments?user.id=096d7e0380e84c97a9b72b4203a19abd&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=85e25458ed46458b8202ba24c3fe7f7e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:30.154976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd45-e4bc-4224-a939-791e4dea7450 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.155559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd45-e4bc-4224-a939-791e4dea7450 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.155559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd45-e4bc-4224-a939-791e4dea7450 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.156943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 617/1247] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:30.162162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.162660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.162749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.167753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:30.179771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:30.181332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:30.248064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 631/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:30.272680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd45-e4bc-4224-a939-791e4dea7450 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:30.292612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:30.375183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd45-e4bc-4224-a939-791e4dea7450 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:30.408691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:30.409412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.409694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.409965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.419847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:30.427474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:30.428605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:30.429798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 618/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:30.435219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:30.533829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ece8-d46e-4a43-a100-9b3136d3a05d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:30.562664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:30.567532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:30.568198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.568515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.568798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.569553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:30.569917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:30.571906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:30.579688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 632/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:30.585451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:30.667346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:30.690316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae74ed1-4ec7-4ac6-ba83-5e1d85f7c489 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:30.724400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:30.725195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.725461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.725734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.736550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:30.737998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:30.755352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:30.756449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:30.757560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:30.757560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:30.757560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:30.757560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:30.757560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 619/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:30.763385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:30.862683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50e70a-772f-476f-bc54-73e13424fa13 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:30.895972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:30.896673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:30.896960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:30.897240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:30.897965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:30.898275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:30.899766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:30.911946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 633/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:30.919003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.018947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d1aca-9337-47b9-bdfe-32fe3f5cfad8 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:31.046229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:31.054104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:31.054906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:31.055231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:31.056296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:31.063778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:31.065253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:31.079684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:31.088700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:31.089676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 620/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:31.090490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.143672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:31.189228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced6a7e-fd91-4083-bbd2-f134424df3bc None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:31.225578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:31.226301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:31.226631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:31.226960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:31.228576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:31.228924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:31.230697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:31.247652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 634/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:31.253769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.355382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3015804f-7a67-48be-94d7-f1e225aa4fb5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:31.387741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:31.388434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:31.388729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:31.389014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:31.444813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:31.458384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:31.460329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:31.560533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 621/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:09:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:09:31.566541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.664545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd33a60-6d7b-494c-8fdb-8d994a29a885 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:31.678258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:31.698896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:31.699632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:31.699921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:31.700202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:31.710599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:31.712684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:31.730276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:31.731185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:31.732161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:31.732161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:31.732161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:31.732161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:31.732465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 635/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:31.737337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.823507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:31.835016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c76a1-4030-4323-b785-f11daabd7aa3 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:31.869899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:31.870554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:31.870837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:31.871114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:31.871836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:31.872141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:31.873700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:31.885843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 622/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:31.891633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:31.992205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d69a2-be66-4f78-a2e6-8d1b7fedfb75 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:32.027607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:32.028504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:32.028796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:32.029081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:32.035755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:32.037262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:32.051105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:32.052142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:32.053102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 636/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:32.059191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:32.158614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b517b0-cace-4e83-a770-14336e3f5239 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:32.191430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:32.192087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:32.192400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:32.192682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:32.193517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:32.193820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:32.195258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9c9d1-33f8-4024-b349-edd30bb6f6f6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:32.207483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 623/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:32.214245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:32.240164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:32.312986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d053-1b42-41cb-b15e-913927b18ef3 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:32.347824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:32.348505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:32.348798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:32.349082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:32.349925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=096d7e0380e84c97a9b72b4203a19abd) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:32.350241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:32.351681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c7923f-8801-42fe-ba42-849a7ae4156d None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:32.357238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 637/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:09:32 2022] GET /identity/v3/role_assignments?user.id=096d7e0380e84c97a9b72b4203a19abd&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:34.387300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e39a3-7e9e-4a11-b800-15fc1d1b012e None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:34.387957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e39a3-7e9e-4a11-b800-15fc1d1b012e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:34.387957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e39a3-7e9e-4a11-b800-15fc1d1b012e None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:34.389229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 624/1261] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:34.394575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:34.401453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:34.401453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:34.412339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:34.413552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:34.488573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 638/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:34.510775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e39a3-7e9e-4a11-b800-15fc1d1b012e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:34.521652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:34.610883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e39a3-7e9e-4a11-b800-15fc1d1b012e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:34.643723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:34.644518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:34.644800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:34.645081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:34.655815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:34.663586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:34.664931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:34.666217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 625/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:34.670943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:34.688968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:34.770879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c970a8-3cbf-46f7-9fdd-dba9d0a7120e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:34.807522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:34.808201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:34.808523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:34.808808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:34.809570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:34.809886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:34.811263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:34.819332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 639/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:34.826126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:34.892225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:34.926501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4343e-d3de-4a93-8c9e-0ea98a40a559 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:34.960746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:34.961594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:34.961853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:34.962316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:34.973728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:34.975191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:34.992477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:34.994068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:34.995545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:34.995545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:34.995545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:34.995545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:34.995545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 626/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:35.000001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.083777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:35.103124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd0545-20bf-4224-bd2a-cc01263b9fb0 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.137022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.138118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.138267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.138539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.139694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:35.140040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.141484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.153668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 640/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:35.159813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.267057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d766b78-003f-4f90-b043-51f5369c8b29 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.286300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:35.300081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.300903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.301223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.301509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.309389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.310834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:35.325374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:35.331617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:35.332850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 627/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:09:35.333617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.433880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa50f3-a519-421b-9291-16e40678da89 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.471616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.474094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.474094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.474094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.474094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:35.474506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.475605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.485683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:35.493833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 641/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:35.500372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.600752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37000f1f-cdd4-49e7-b1b4-750072de1422 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.634480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.635298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.635583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.635858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.646263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.651745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:35.665117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:35.666814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:35.667843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:35.667843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:35.667843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:35.667843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:35.667843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 628/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:35.672580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.769530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87192dbf-3e9c-4675-8ef4-638025055157 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.804812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.805458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.805773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.806052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.806974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:35.807279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.808683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.820125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 642/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:35.826023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:35.860338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:35.926119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fad6a5-336a-42a3-a54a-a7b947e4ed90 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:35.959018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:35.959808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:35.960094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:35.960419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:35.968502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:35.969960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:35.984515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:35.987369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:35.988541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 629/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:09:35.991843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:36.053007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:36.094862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb437e7-884d-49da-a122-ca132def4194 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:36.127406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:36.127999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:36.128286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:36.128593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:36.129379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:36.129737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:36.131162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a115873-2db0-48da-ba04-0f422ecfd406 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:36.143462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 643/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:36.149556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:36.240430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:36.249275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880cdd4-b44f-4647-9369-5d5e2c876785 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:36.283567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:36.284493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:36.284777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:36.285055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] PATH_INFO: `/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/096d7e0380e84c97a9b72b4203a19abd/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:36.287011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] RBAC: Authorizing `identity:create_grant(project_id=d09adfd8fd6e4533ba29be1b6222468b, user_id=096d7e0380e84c97a9b72b4203a19abd, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:36.288486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3611f-8f72-4e54-9010-2e912257a86e None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:36.411608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 630/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Tue Jul 12 18:09:36 2022] PUT /identity/v3/projects/d09adfd8fd6e4533ba29be1b6222468b/users/096d7e0380e84c97a9b72b4203a19abd/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:09:38.444622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d1e3-d7c9-4881-8541-1ce1ebf252a2 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:38.445186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d1e3-d7c9-4881-8541-1ce1ebf252a2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:38.445186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4d1e3-d7c9-4881-8541-1ce1ebf252a2 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:38.446682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 644/1274] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:38.452553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:38.452991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:38.453103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:38.470871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:38.472391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:38.538194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 631/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:38.561324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d1e3-d7c9-4881-8541-1ce1ebf252a2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:38.663752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4d1e3-d7c9-4881-8541-1ce1ebf252a2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:38.680662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:38.699880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:38.700642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:38.700928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:38.701208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:38.711121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 18:09:38.718928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:38.720045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:38.721285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 645/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Tue Jul 12 18:09:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:38.726338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:38.809082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:38.827011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85692e7f-0cf2-4cf9-9432-cf1c4ce502c7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:38.860733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:38.861504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:38.861794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:38.862143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:38.863218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:38.863537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:38.865343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:38.876562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 632/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Tue Jul 12 18:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:38.882278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:38.984164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e1918-c326-4cd8-b482-47746c4d5c06 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:39.017928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:39.018758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:39.019095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:39.019380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:39.029858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:39.031431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:39.048991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:39.050111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:39.051058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:39.051058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:39.051058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:39.051058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:39.051058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 646/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:39.056073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:39.155863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2d044-b77b-4603-b075-a08a3817b897 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:39.189322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:39.189984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:39.190270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:39.190547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:39.191277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:39.191584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:39.193026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:39.204612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 633/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:39.210324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:39.241732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:39.311343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acf906-0e2a-4911-9e98-d4b647500e70 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:39.344689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:39.345489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:39.345776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:39.346058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:39.354082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:39.355539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 18:09:39.370675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:39.371812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:39.373299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 647/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:09:39.377712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:39.381499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:39.476293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1c5a3-2a1d-4de1-8199-4aaa1eb4c95a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:39.509751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:39.510360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:39.510651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:39.510935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:39.511821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:39.512203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:39.513706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:39.531111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 634/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Tue Jul 12 18:09:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:09:39.536897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:39.628645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:39.638040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f86cce-3d94-4558-8604-2e05649f36fe None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:39.672367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:39.673024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:39.673313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:39.673597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:39.740932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:39.742433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:39.843966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:39.860636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 648/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Tue Jul 12 18:09:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 18:09:39.861728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:39.929333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:39.971436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad7662-9354-49c9-8a72-8ff13240ba7a None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:40.014926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:40.015744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:40.016033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:40.016345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:40.028823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:40.030350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 18:09:40.048271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:40.054401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:40.055428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 18:09:40.055428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 18:09:40.055428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 18:09:40.055428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:40.055428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 635/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:40.055737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:40.165057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863723b-bbe2-4ff3-8262-b147ac118723 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:40.200370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:40.201001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:40.201287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:40.201569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:40.202288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:40.202595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:40.204030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:40.215605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 649/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Tue Jul 12 18:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:40.221384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:40.319160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06005fc7-8e5f-4631-91f2-21645a19a6c5 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:40.333048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:40.352576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:40.353498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:40.353780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:40.354060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:40.360650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:40.362103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 18:09:40.376770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:40.377737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 18:09:40.378656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: ERROR keystone.server.flask.application Jul 12 18:09:40.379513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 636/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Tue Jul 12 18:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:40.383737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:40.433677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:40.485608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c52988-f617-44a0-954e-bbf2394501f5 None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:40.518746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:40.519403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:40.519706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:40.519973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:40.520790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:40.521101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:40.522525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd1132-66f1-4baf-a392-a782433f87ab None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:40.534442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 650/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Tue Jul 12 18:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:40.540631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:40.639317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0286a-f735-4b25-8e1a-e56714ed3fec None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:40.672327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:40.672999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:40.673289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:40.673571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:40.674406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] RBAC: Adding query filter params (role.id=be70b8e955f74d66a3dd5d81b794548e, scope.project.id=d09adfd8fd6e4533ba29be1b6222468b, user.id=096d7e0380e84c97a9b72b4203a19abd) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:40.674719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:40.676190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402050c7-b81f-4e17-9d35-c3062362a47f None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:40.682577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 637/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Tue Jul 12 18:09:40 2022] GET /identity/v3/role_assignments?user.id=096d7e0380e84c97a9b72b4203a19abd&scope.project.id=d09adfd8fd6e4533ba29be1b6222468b&role.id=be70b8e955f74d66a3dd5d81b794548e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:42.723930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cc7ad-445d-4ed9-adec-074e38374680 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:42.724454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cc7ad-445d-4ed9-adec-074e38374680 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:42.724454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cc7ad-445d-4ed9-adec-074e38374680 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:42.725818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 651/1288] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:42.730897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:42.731326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:42.731475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:42.748612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:42.749838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:42.779401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:42.815671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 638/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:42.838478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cc7ad-445d-4ed9-adec-074e38374680 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:42.940314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cc7ad-445d-4ed9-adec-074e38374680 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:42.951179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:42.973423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:42.974263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:42.974497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:42.974773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] PATH_INFO: `/v3/services/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:42.975527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:42.976873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:42.989150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:42.990424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:42.991368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 652/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:09:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:42.997250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:43.094854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2180c9-1867-4362-8814-e9271a701ad6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:43.128518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:43.129183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:43.129471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:43.129872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:43.130515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:43.130828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:43.132230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003cebab-be75-4e93-858f-943ad7ab5713 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:43.137556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 639/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Tue Jul 12 18:09:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:43.143885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:43.246207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686d2ab-1ebb-4aea-adab-23ba481a64ca None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:43.279256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:43.279915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:43.280206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:43.280515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] PATH_INFO: `/v3/services` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:43.281224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] RBAC: Adding query filter params (type=placement) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:43.281565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:43.282936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5a172-2a7e-4a27-afb5-4e5f85a3d3fd None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:43.287460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 653/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Tue Jul 12 18:09:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:45.320252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783a37-80bd-41c1-938c-7e0ad15c96f7 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:45.320839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783a37-80bd-41c1-938c-7e0ad15c96f7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:45.320839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18783a37-80bd-41c1-938c-7e0ad15c96f7 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:45.322226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 640/1293] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:45.328135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1963e-60b7-4028-bbdf-81cdece7d773 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:45.328571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1963e-60b7-4028-bbdf-81cdece7d773 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:45.328751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1963e-60b7-4028-bbdf-81cdece7d773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:45.346167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1be1963e-60b7-4028-bbdf-81cdece7d773 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:45.347681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1be1963e-60b7-4028-bbdf-81cdece7d773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:45.378544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:45.414533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 654/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:45.437336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18783a37-80bd-41c1-938c-7e0ad15c96f7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:45.474137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:45.545422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18783a37-80bd-41c1-938c-7e0ad15c96f7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:45.579808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:45.580421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:45.580707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:45.580990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:45.581727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:45.583156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89172d-c77a-4bf0-a37a-a9bffc87d09d None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:45.696753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 641/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Tue Jul 12 18:09:45 2022] POST /identity/v3/services => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:09:47.739057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d66f4-400a-4e3e-aade-1872d2629be6 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:47.739575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d66f4-400a-4e3e-aade-1872d2629be6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:47.739575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d66f4-400a-4e3e-aade-1872d2629be6 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:47.740984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 655/1296] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:47.746622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:47.747021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:47.747176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:47.764379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:47.765603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:47.837422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 642/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:47.859945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d66f4-400a-4e3e-aade-1872d2629be6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:47.965128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d66f4-400a-4e3e-aade-1872d2629be6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:47.999467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:48.000143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:48.000456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:48.000839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:48.001526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:48.002902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:48.015561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:48.019214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:48.020212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 656/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:09:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 18:09:48.022870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:48.042270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:48.122447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb4b3-0080-48d7-8d43-ba14fb31f24c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:48.156693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:48.157360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:48.157650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:48.157931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:48.158651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:48.158963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:48.160378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:48.165555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 643/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Tue Jul 12 18:09:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:48.172034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:48.203544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:48.270181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafecd25-df89-4c32-bc68-c00a9ee5fc3f None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:48.303516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:48.304170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:48.304487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:48.304766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:48.305555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] RBAC: Adding query filter params (interface=public, service_id=9bf4adbe0cae405ea0827506dd530d1d, region_id=RegionOne) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:48.305865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:48.307257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d5218-43d3-4d75-b6f5-27b50f4bc031 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:48.312902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 657/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Tue Jul 12 18:09:48 2022] GET /identity/v3/endpoints?service_id=9bf4adbe0cae405ea0827506dd530d1d&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:48.318601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:48.399363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:48.416967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a1439-dfea-433f-bc0f-d5129089dcb6 None admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:48.449963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:48.450631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:48.450915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:48.451192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:48.451879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:48.453307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa970f90-5ae6-4be0-b3bc-04b3e43d3335 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:48.460291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 644/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Tue Jul 12 18:09:48 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:09:50.457496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaebe5-5447-49a5-ad36-6dd797040400 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:50.457952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaebe5-5447-49a5-ad36-6dd797040400 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:50.458065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaebe5-5447-49a5-ad36-6dd797040400 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:50.461653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 658/1302] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:09:50.465153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:50.465546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:50.465710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:50.482824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:09:50.484040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:09:50.550535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 645/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:09:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 18:09:50.572952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaebe5-5447-49a5-ad36-6dd797040400 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:50.667490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:50.674133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaebe5-5447-49a5-ad36-6dd797040400 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:50.707444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:50.708162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:50.708483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:50.708765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:50.709416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:50.710781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 18:09:50.723485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 18:09:50.724675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application with Lock( Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self._enter() Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application return self.creator() Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application created_value = creator( Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: ERROR keystone.server.flask.application Jul 12 18:09:50.725609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 659/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Tue Jul 12 18:09:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 18:09:50.730917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:50.805111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:09:50.828209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58039f54-89e0-4bad-a466-4a06d916835a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:50.861833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:50.862501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:50.862789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:50.863072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] PATH_INFO: `/v3/services` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:50.863789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:09:50.864103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:50.865531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db122966-8e4d-4be5-863b-9ef838872212 None admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:50.871452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 646/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Tue Jul 12 18:09:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:09:50.877935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:09:50.980620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27b96d-9d62-4012-8c30-a5c8425a5e9e None admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:09:51.013605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:09:51.014253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:09:51.014538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:09:51.014817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:09:51.015601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:09:51.017012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a9408-fbaa-4985-ba01-ea5f4e6315b6 None admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:09:51.113621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 660/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Tue Jul 12 18:09:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:10:00.885018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=30 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:10:01.108959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:36.764624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf96a3-76cf-4ff0-819d-7a945fe1bfd1 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:36.765206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf96a3-76cf-4ff0-819d-7a945fe1bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:36.765206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf96a3-76cf-4ff0-819d-7a945fe1bfd1 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:36.771891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d9912-e445-4958-b262-93d7cdb37efb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:36.772394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d9912-e445-4958-b262-93d7cdb37efb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:36.772506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d9912-e445-4958-b262-93d7cdb37efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:36.774641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 647/1307] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:36.860462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-476d9912-e445-4958-b262-93d7cdb37efb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:36.861754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-476d9912-e445-4958-b262-93d7cdb37efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:37.004958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 661/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:16:37.044694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:37.044992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:37.045277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:37.046954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 648/1309] 10.222.0.47 () {54 vars in 964 bytes} [Tue Jul 12 18:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:37.053970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987b2d-8fc2-4ff2-afd8-892d2f35cda2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:37.054269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987b2d-8fc2-4ff2-afd8-892d2f35cda2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:37.054573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20987b2d-8fc2-4ff2-afd8-892d2f35cda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:37.193241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-20987b2d-8fc2-4ff2-afd8-892d2f35cda2 None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:37.194826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-20987b2d-8fc2-4ff2-afd8-892d2f35cda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:37.226202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:37.311081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 662/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Tue Jul 12 18:16:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:37.321053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:37.323296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:37.451642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cc96e-3e9d-4cb6-bec1-a1a466cb7156 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:37.530928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:37.531595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:37.531882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:37.532161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:37.533914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:37.591733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:37.646851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:37.648384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10809e9-b4de-4ec8-9d79-07c459876259 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:37.669708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 649/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:40.012131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:40.012705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:40.012705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:40.014090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 663/1312] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:40.019479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc635a3-8735-4b81-970b-0a41a76561e1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:40.019905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc635a3-8735-4b81-970b-0a41a76561e1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:40.020167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc635a3-8735-4b81-970b-0a41a76561e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:40.092839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9fc635a3-8735-4b81-970b-0a41a76561e1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:40.094375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9fc635a3-8735-4b81-970b-0a41a76561e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:40.198405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 650/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:40.237499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:40.239779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:40.358670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b6971-d490-4040-b1fd-94bf60a2e77c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:40.436123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:40.438978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:40.439779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:40.440256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:40.440480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:40.442298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:40.584796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:40.586420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fd9af-ec82-44c3-8657-0a193d6353ef service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:40.608949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 664/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:43.133068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:43.133607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:43.133702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:43.135568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 651/1315] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:43.140707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62180e3-99b4-496d-bcd9-bb9a5f0df4c9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:43.141110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62180e3-99b4-496d-bcd9-bb9a5f0df4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:43.141280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62180e3-99b4-496d-bcd9-bb9a5f0df4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:43.208658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b62180e3-99b4-496d-bcd9-bb9a5f0df4c9 None None] MFA Rules not processed for user `2a598a92f6ea439a98503435cf969703`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:43.209876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b62180e3-99b4-496d-bcd9-bb9a5f0df4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:43.262958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:43.309686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 665/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:16:43.349294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:43.351017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:43.363049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:43.462375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533fb82c-c34c-4145-b5f4-119ee5231419 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:43.535775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:43.536488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:43.536778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:43.537062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:43.538743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:43.601518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:43.650107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:43.651568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c91a95-5663-44b1-9363-ef53971c4b07 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:43.670716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 652/1317] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:46.922426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:46.923053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:46.923053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:46.924404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 666/1318] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:46.930169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aac0-b260-4778-bbaa-c994da230fd5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:46.930699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aac0-b260-4778-bbaa-c994da230fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:46.930699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aac0-b260-4778-bbaa-c994da230fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:46.996354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6416aac0-b260-4778-bbaa-c994da230fd5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:46.997575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6416aac0-b260-4778-bbaa-c994da230fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:47.094408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 653/1319] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:47.136608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:47.138164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:47.242562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700fbfe-d129-4bff-b85f-6c4d16c57fd5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:47.317687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:47.317950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:47.318202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:47.318480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:47.320174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:47.328218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:47.432730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:47.434239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308ebdab-4b47-49f5-9a1b-8205f1519555 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:47.453655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 667/1320] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:50.659709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:50.660240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:50.662891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:50.662891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 654/1321] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:50.667337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d06ed-68da-471b-ad24-656a08df43a9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:50.667797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d06ed-68da-471b-ad24-656a08df43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:50.667933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d06ed-68da-471b-ad24-656a08df43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:50.735156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-781d06ed-68da-471b-ad24-656a08df43a9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:50.736299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-781d06ed-68da-471b-ad24-656a08df43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:50.796711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:50.838206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 668/1322] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:50.881527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:50.886252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:50.911610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:51.011299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2330c-24b9-40f9-bce7-bafb7daefe89 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:51.093813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:51.094474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:51.094764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:51.095045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:51.096744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:51.173925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:51.219805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:51.221300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0545406-aa36-4472-b71b-1dc1c4598763 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:51.240336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 655/1323] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:16:53.390319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937c606-7752-496b-8cd8-e5e034f2b3ff None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:53.390972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937c606-7752-496b-8cd8-e5e034f2b3ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:53.390972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937c606-7752-496b-8cd8-e5e034f2b3ff None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:53.392234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 669/1324] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:53.397980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea74b-0cd6-436d-b056-88799376dbcd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:53.398498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea74b-0cd6-436d-b056-88799376dbcd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:53.398688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eea74b-0cd6-436d-b056-88799376dbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:53.472194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-91eea74b-0cd6-436d-b056-88799376dbcd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:53.473453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-91eea74b-0cd6-436d-b056-88799376dbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:53.579261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 656/1325] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:16:53.629155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:53.629474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:53.629764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:53.631265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 670/1326] 10.222.0.47 () {54 vars in 964 bytes} [Tue Jul 12 18:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:53.638324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f48647-0930-4d2c-809a-db7a6f58273c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:53.638725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f48647-0930-4d2c-809a-db7a6f58273c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:53.639079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f48647-0930-4d2c-809a-db7a6f58273c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:53.670742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:53.719738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-55f48647-0930-4d2c-809a-db7a6f58273c None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:53.720905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-55f48647-0930-4d2c-809a-db7a6f58273c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:53.801738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 657/1327] 10.222.0.47 () {58 vars in 1061 bytes} [Tue Jul 12 18:16:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:53.811783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:53.813355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:53.924906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f125986-1e32-48b2-aab4-6aabcb7cb0b3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:54.002319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:54.002844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:54.003152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:54.003410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:54.005129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:54.022556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:54.130166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:54.131612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5380d80-bf57-4a8f-8d28-7099569392f3 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:54.151197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 671/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:56.557255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:56.557878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:56.557878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:56.559266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 658/1329] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:56.564660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b36e12-37b9-4f4c-b97c-313c221856b9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:56.565091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b36e12-37b9-4f4c-b97c-313c221856b9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:56.565238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b36e12-37b9-4f4c-b97c-313c221856b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:56.631899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e9b36e12-37b9-4f4c-b97c-313c221856b9 None None] MFA Rules not processed for user `2a598a92f6ea439a98503435cf969703`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:56.633155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e9b36e12-37b9-4f4c-b97c-313c221856b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:56.712843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:56.752900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 672/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:16:56.791819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:16:56.794086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:56.900555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cf45d-13ae-48a1-b23f-2fb3047e62b8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:16:56.920973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:56.975807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:16:56.976417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:56.976713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:56.976993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:56.979450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:16:57.091222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:16:57.092720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbce09b-1088-421e-9d76-594e03524880 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:16:57.111497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 659/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:16:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:16:59.849724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:59.850255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:59.850255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:59.851606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 673/1332] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:16:59.857642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b5c41-b511-4e54-a0a1-ca9edd3018a4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:16:59.858082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b5c41-b511-4e54-a0a1-ca9edd3018a4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:16:59.858208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b5c41-b511-4e54-a0a1-ca9edd3018a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:16:59.919910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:16:59.922804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4a5b5c41-b511-4e54-a0a1-ca9edd3018a4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:16:59.924028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4a5b5c41-b511-4e54-a0a1-ca9edd3018a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:00.019138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 660/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:00.055323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:00.057094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:00.160839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4965d-0f01-4a16-a1f8-c97e4f45385e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:00.235107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:00.235761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:00.236053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:00.236359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:00.238058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:00.266283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:00.347621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:00.349089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adeb2f-85e9-4747-a19d-3bd008f491f0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:00.368113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 674/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:02.523736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:02.524260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:02.524405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:02.525699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 661/1335] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:02.530812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2cea-4c00-47ae-9ced-168b2d1993b7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:02.531267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2cea-4c00-47ae-9ced-168b2d1993b7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:02.531364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb2cea-4c00-47ae-9ced-168b2d1993b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:02.602231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-42eb2cea-4c00-47ae-9ced-168b2d1993b7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:02.603614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-42eb2cea-4c00-47ae-9ced-168b2d1993b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:02.680252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:02.703562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 675/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:02.743041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:02.745756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:02.849141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c01fb-b450-4a29-b175-9c7ef2342a68 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:02.878014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:02.922796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:02.923463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:02.923937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:02.924430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:02.926480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:03.035325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:03.036768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e109a3-86fa-415c-ab7b-f719b7f6add1 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:03.056342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 662/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:05.881999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:05.882536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:05.882536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:05.883887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 676/1338] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:17:05.889308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6574b-fe56-4e70-ad18-1a8f39a30376 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:05.889717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6574b-fe56-4e70-ad18-1a8f39a30376 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:05.889880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6574b-fe56-4e70-ad18-1a8f39a30376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:05.953191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c6b6574b-fe56-4e70-ad18-1a8f39a30376 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:05.954405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c6b6574b-fe56-4e70-ad18-1a8f39a30376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:05.962535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:06.051524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 663/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:06.088285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:06.089865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:06.201826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec394a7-402f-4967-b769-df88b2849ccc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:06.279270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:06.279930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:06.280221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:06.280548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:06.282225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:06.318664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:06.393570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:06.395015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb788d-b9af-46d4-8a68-c55b8d3d79eb service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:06.414241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 677/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:10.785030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:10.785510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:10.785594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:10.786997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 664/1341] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:10.792646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8401-f3eb-49ee-9d49-2cba70efb47d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:10.793177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8401-f3eb-49ee-9d49-2cba70efb47d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:10.793266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8401-f3eb-49ee-9d49-2cba70efb47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:10.870409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d35b8401-f3eb-49ee-9d49-2cba70efb47d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:10.871684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d35b8401-f3eb-49ee-9d49-2cba70efb47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:10.982141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:10.995323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 678/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:11.034593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:11.036067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:11.145499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ea4da-5f0f-45c1-aa0b-2966e9f7f437 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:11.188581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:11.227875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:11.228764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:11.229054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:11.229333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:11.231500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:11.364632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:11.366047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76f965-e7eb-45bf-87fb-aaa43b9814c5 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:11.386532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 665/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:14.583789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:14.584342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:14.584342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:14.585672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 679/1344] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:14.591206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d93dd-427f-4431-8d3b-d8a7ddb6151d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:14.591645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d93dd-427f-4431-8d3b-d8a7ddb6151d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:14.591907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d93dd-427f-4431-8d3b-d8a7ddb6151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:14.661237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d64d93dd-427f-4431-8d3b-d8a7ddb6151d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:14.662445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d64d93dd-427f-4431-8d3b-d8a7ddb6151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:14.680998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:14.766028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 666/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:14.803908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:14.806021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:14.920012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18d3b-7438-4498-912e-34260d6865ea None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:14.997042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:14.997705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:14.997998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:14.998285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:14.999973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:15.050100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:15.119685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:15.121167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237aadb5-deb0-44c9-bccc-d9fc81a66d6a service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:15.139581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 680/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:19.878161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:19.878726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:19.878824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:19.880333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 667/1347] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:19.886368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801704c-df6c-470e-b8f1-cafbcc18223a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:19.886904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801704c-df6c-470e-b8f1-cafbcc18223a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:19.887001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801704c-df6c-470e-b8f1-cafbcc18223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:19.959347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d801704c-df6c-470e-b8f1-cafbcc18223a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:19.960633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d801704c-df6c-470e-b8f1-cafbcc18223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:20.063008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:20.065644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 681/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:20.104352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:20.106247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:20.220130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10ed00-afb1-4fc8-9276-841685f2a14a None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:20.270479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:20.300084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:20.300883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:20.301460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:20.301746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:20.303434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:20.421270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:20.422742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b22824-15bb-4949-bcde-78d8c547a497 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:20.442200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 668/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:25.445876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065565-c9b2-465e-9e25-33b0a26ab205 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:25.446827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065565-c9b2-465e-9e25-33b0a26ab205 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:25.447400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3065565-c9b2-465e-9e25-33b0a26ab205 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:25.449298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 682/1350] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:17:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:25.454484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c743c-f663-46f8-a61f-e7362ec96e7f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:25.454962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c743c-f663-46f8-a61f-e7362ec96e7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:25.455075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c743c-f663-46f8-a61f-e7362ec96e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:25.588476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3e2c743c-f663-46f8-a61f-e7362ec96e7f None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:25.590042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3e2c743c-f663-46f8-a61f-e7362ec96e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:25.631954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:25.735523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 669/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:17:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:29.981456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04534-0303-4067-b7f5-9f8e6ab078b0 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:29.982010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04534-0303-4067-b7f5-9f8e6ab078b0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:29.982094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d04534-0303-4067-b7f5-9f8e6ab078b0 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:29.983601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 683/1352] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:17:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:29.988464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efb7d-e885-4664-bb81-4bbd8185ed6c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:29.989302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efb7d-e885-4664-bb81-4bbd8185ed6c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:29.990452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80efb7d-e885-4664-bb81-4bbd8185ed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:30.070363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c80efb7d-e885-4664-bb81-4bbd8185ed6c None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:30.071959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c80efb7d-e885-4664-bb81-4bbd8185ed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:30.149958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:30.189063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 670/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:17:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:31.560983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5000-a8ff-4da4-ad94-1b0515e3c90f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:31.561509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5000-a8ff-4da4-ad94-1b0515e3c90f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:31.561628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f5000-a8ff-4da4-ad94-1b0515e3c90f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:31.562908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 684/1354] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:31.567865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b27d-3bf5-4967-a70a-bd253b8ece43 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:31.568284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b27d-3bf5-4967-a70a-bd253b8ece43 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:31.568553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655b27d-3bf5-4967-a70a-bd253b8ece43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:31.645894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5655b27d-3bf5-4967-a70a-bd253b8ece43 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:31.647104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5655b27d-3bf5-4967-a70a-bd253b8ece43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:31.757695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 671/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:17:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:35.259824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb5fb-92a0-4bd3-bc92-2f28c6fa015a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:35.260703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb5fb-92a0-4bd3-bc92-2f28c6fa015a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:35.261613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bb5fb-92a0-4bd3-bc92-2f28c6fa015a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:35.263991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 685/1356] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:35.269288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17805-157e-48d0-b23f-347f7e2dce0e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:35.270194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17805-157e-48d0-b23f-347f7e2dce0e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:35.270851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17805-157e-48d0-b23f-347f7e2dce0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:35.321764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:35.373936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dba17805-157e-48d0-b23f-347f7e2dce0e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:35.375310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dba17805-157e-48d0-b23f-347f7e2dce0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:35.503560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 672/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:35.560579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:35.561138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:35.561606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:35.563299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 686/1358] 10.222.0.47 () {54 vars in 961 bytes} [Tue Jul 12 18:17:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:35.571628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882119f6-9c8b-4bc0-aca5-fee9897d4a64 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:35.571971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882119f6-9c8b-4bc0-aca5-fee9897d4a64 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:35.572253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882119f6-9c8b-4bc0-aca5-fee9897d4a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:35.660294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-882119f6-9c8b-4bc0-aca5-fee9897d4a64 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:35.661708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-882119f6-9c8b-4bc0-aca5-fee9897d4a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:35.721580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:35.816460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 673/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Tue Jul 12 18:17:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:35.827025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:35.828828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:35.990146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceba7c9-740c-43a0-a198-da799e80bad8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:36.088279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:36.141071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:36.141675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:36.141970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:36.142249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:36.143957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:36.245618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:36.247404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a70bf-14f8-4b5b-a4a4-ebb12638bc17 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:36.268969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 687/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:38.581373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:38.581888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:38.582354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:38.583907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 674/1361] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:38.589887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5428c-9f20-4ea3-af8f-a7de5685807b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:38.590894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5428c-9f20-4ea3-af8f-a7de5685807b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:38.591335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5428c-9f20-4ea3-af8f-a7de5685807b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:38.661332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f9b5428c-9f20-4ea3-af8f-a7de5685807b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:38.662565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f9b5428c-9f20-4ea3-af8f-a7de5685807b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:38.701426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:38.768555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 688/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:38.810104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:38.811598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:39.001069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb7a1-a809-4e67-99d8-f919cbc61677 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:39.012683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:39.178538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:39.179192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:39.179479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:39.180277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:39.181965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:39.278313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:39.279988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2121e-2085-42f2-b033-954e86930d67 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:39.300861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 675/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:41.931913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5aae3-9426-402d-b73a-9c2be09018a3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:41.932509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5aae3-9426-402d-b73a-9c2be09018a3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:41.932509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa5aae3-9426-402d-b73a-9c2be09018a3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:41.933904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 689/1364] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:41.939291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778ea4-1ec0-49d5-9615-f92b9ddd390b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:41.940109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778ea4-1ec0-49d5-9615-f92b9ddd390b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:41.940455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a778ea4-1ec0-49d5-9615-f92b9ddd390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:41.956697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:42.019637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9a778ea4-1ec0-49d5-9615-f92b9ddd390b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:42.021068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9a778ea4-1ec0-49d5-9615-f92b9ddd390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:42.131992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 676/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:42.182538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:42.182810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:42.183044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:42.184657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 690/1366] 10.222.0.47 () {54 vars in 961 bytes} [Tue Jul 12 18:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:42.190265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000f1d-9933-4a89-9f84-2db4c48d5fb2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:42.190616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000f1d-9933-4a89-9f84-2db4c48d5fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:42.190899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d000f1d-9933-4a89-9f84-2db4c48d5fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:42.275259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2d000f1d-9933-4a89-9f84-2db4c48d5fb2 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:42.276726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2d000f1d-9933-4a89-9f84-2db4c48d5fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:42.316665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:42.415551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 677/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Tue Jul 12 18:17:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:42.423476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:42.425147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:42.744428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa294-836f-4ef5-8094-b831084b7b4b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:42.769804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:42.871081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:42.871742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:42.872032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:42.872341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:42.874038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:42.970926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:42.972574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892727b2-a18b-40d8-a7d4-fb11fb3617d0 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:42.992520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 691/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:43.711608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97238c-a955-431c-937c-8d0269096b12 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:43.712191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97238c-a955-431c-937c-8d0269096b12 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:43.712290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97238c-a955-431c-937c-8d0269096b12 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:43.713822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 678/1369] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:43.719034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defb79-a98f-4e7e-81da-19f078a5346d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:43.719334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defb79-a98f-4e7e-81da-19f078a5346d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:43.719621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7defb79-a98f-4e7e-81da-19f078a5346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:43.754112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:43.792459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d7defb79-a98f-4e7e-81da-19f078a5346d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:43.793693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d7defb79-a98f-4e7e-81da-19f078a5346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:43.899960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 692/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:44.252974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:44.253270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:44.253594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:44.255115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 679/1371] 10.222.0.47 () {54 vars in 963 bytes} [Tue Jul 12 18:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:44.260500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d48db5-dff9-465d-94fa-60721c5d994f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:44.261034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d48db5-dff9-465d-94fa-60721c5d994f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:44.261186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d48db5-dff9-465d-94fa-60721c5d994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:44.345079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a6d48db5-dff9-465d-94fa-60721c5d994f None None] MFA Rules not processed for user `44fe9acbbbdc463ab22869886df4e3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:44.346313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a6d48db5-dff9-465d-94fa-60721c5d994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:44.402865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:44.435350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 693/1372] 10.222.0.47 () {58 vars in 1060 bytes} [Tue Jul 12 18:17:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:44.444146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:44.444534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:44.572431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a40325-4c99-4250-91a8-3e80e4126d8b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:44.615693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:44.650993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:44.651649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:44.651938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:44.652220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:44.653944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:44.774270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:44.776497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5158e6-db8b-4ff9-8032-724b74649bd4 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:44.800446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 680/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:45.331682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:45.332208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:45.332208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:45.333549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 694/1374] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:45.338781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d338e-56c9-41ec-b26f-476ac647b9f5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:45.339076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d338e-56c9-41ec-b26f-476ac647b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:45.339360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d338e-56c9-41ec-b26f-476ac647b9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:45.408114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4a2d338e-56c9-41ec-b26f-476ac647b9f5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:45.409366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4a2d338e-56c9-41ec-b26f-476ac647b9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:45.431688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:45.513865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 681/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:45.558450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:45.560093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:45.724871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd89c-0562-4636-9704-c4d2dd1e0117 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:45.780123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:45.843233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:45.843889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:45.844181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:45.844498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:45.846173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:45.942026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:45.943639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c4930-7a4f-48ba-8278-64859a7d547b service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:45.962672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 695/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:48.197594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:48.198181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:48.198181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:48.199514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 682/1377] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:48.205241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bdb-d19d-4a8e-ae2f-9d2f36650f96 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:48.205752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bdb-d19d-4a8e-ae2f-9d2f36650f96 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:48.205872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25bdb-d19d-4a8e-ae2f-9d2f36650f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:48.272318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-72a25bdb-d19d-4a8e-ae2f-9d2f36650f96 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:48.274252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-72a25bdb-d19d-4a8e-ae2f-9d2f36650f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:48.281429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:48.381556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 696/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:48.435627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:48.437220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:48.637071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bcdd2-fe53-4104-8e14-fbd4e79e4f5f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:48.641207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:48.762279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:48.762946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:48.763305 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:48.763658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:48.765434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:48.865093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:48.866870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25702c0e-27f4-4799-8878-b94e7cdcb09a service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:48.886983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 683/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:51.161395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:51.162122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:51.162409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:51.164938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 697/1380] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:51.171045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e289ad5-42ac-4f7f-bdac-27fc1cc13e47 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:51.171045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e289ad5-42ac-4f7f-bdac-27fc1cc13e47 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:51.171045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e289ad5-42ac-4f7f-bdac-27fc1cc13e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:51.182043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:51.247762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0e289ad5-42ac-4f7f-bdac-27fc1cc13e47 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:51.249040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0e289ad5-42ac-4f7f-bdac-27fc1cc13e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:51.364479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 684/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:51.401261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:51.403206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:51.580720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:51.592768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80634b00-504e-413b-bbb2-de1cf7d32f70 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:51.720718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:51.721321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:51.721608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:51.721888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:51.723544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:51.823731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:51.825638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa916485-ba98-4cde-89b8-9cac552dd631 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:51.838454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:51.848850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 698/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:54.397009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:54.397557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:54.397976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:54.400423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 685/1383] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:17:54.407548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eaf25-dd3e-40dc-b09b-a146e352f221 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:54.408012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eaf25-dd3e-40dc-b09b-a146e352f221 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:54.408157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637eaf25-dd3e-40dc-b09b-a146e352f221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:54.483107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-637eaf25-dd3e-40dc-b09b-a146e352f221 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:54.484447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-637eaf25-dd3e-40dc-b09b-a146e352f221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:54.596678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 699/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:54.641772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:54.643363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:54.734192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:54.808896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c58bdc-2516-4a51-b200-4d5a3868eb94 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:54.935538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:54.936421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:54.936806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:54.937403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:54.939780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:55.002887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:55.055472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:55.057773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fa912-c0da-4867-95b8-141dfba45ee8 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:55.083242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 686/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:17:57.666428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:57.666972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:57.667118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:57.668878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 700/1386] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:17:57.675444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde50543-8066-4348-a438-6c090f593c4e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:57.675962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde50543-8066-4348-a438-6c090f593c4e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:57.676234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde50543-8066-4348-a438-6c090f593c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:57.762028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fde50543-8066-4348-a438-6c090f593c4e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:17:57.763603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fde50543-8066-4348-a438-6c090f593c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:57.891464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 687/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:17:57.946750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:17:57.948373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:58.037348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:58.121236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac15e52-0e1d-4ae5-b2af-bc6ad40677c9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:17:58.268264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:17:58.269048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:17:58.269405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:17:58.269906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:17:58.272053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:17:58.321244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:17:58.374803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:17:58.376728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6cde5-1f13-4017-82f3-7f5bd17a5968 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:17:58.395813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 701/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:00.407653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93a06f-0dff-4b92-ad02-dd446b70755c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.408193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93a06f-0dff-4b92-ad02-dd446b70755c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.408406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93a06f-0dff-4b92-ad02-dd446b70755c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.409683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 688/1389] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:00.415662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abb44-82b6-4021-8c95-9be51462202e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.416107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abb44-82b6-4021-8c95-9be51462202e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.416223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abb44-82b6-4021-8c95-9be51462202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.486158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ce4abb44-82b6-4021-8c95-9be51462202e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:00.487388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ce4abb44-82b6-4021-8c95-9be51462202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:00.594320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 702/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:00.602038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8776f3-55fd-4525-855c-43423ceac876 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.602416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8776f3-55fd-4525-855c-43423ceac876 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.602703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8776f3-55fd-4525-855c-43423ceac876 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.604219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 689/1391] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:00.609734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.610031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.610315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.626876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:00.656528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1f49-762e-4a39-9578-a5280dd24421 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.656829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1f49-762e-4a39-9578-a5280dd24421 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.657109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa1f49-762e-4a39-9578-a5280dd24421 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.659603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 690/1392] 10.222.0.47 () {54 vars in 962 bytes} [Tue Jul 12 18:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:00.665537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:00.666806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:00.667259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:00.703016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:00.703515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:00.704984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:00.759289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] MFA Rules not processed for user `b32ef598390145d1b88c3e3811edb79b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:00.760682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:00.820709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 703/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:18:00.852390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 691/1394] 10.222.0.47 () {58 vars in 1059 bytes} [Tue Jul 12 18:18:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:18:00.859982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:00.861276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:00.868658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:00.870510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:00.954560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9bbf2-4e25-434d-91b9-b1321931ddbf None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:00.962658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:00.991828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2036d-aa05-46ef-8deb-cfda99d107bb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:00.995691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:01.031518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:01.032181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:01.032500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:01.032783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:01.034465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:01.108870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:01.109622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:01.109906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:01.110182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:01.111826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:01.151367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:01.152851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d14b6-9d38-4116-ac07-1a53b4fba48e service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:01.173809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 704/1395] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:01.212705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:01.214426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a30a02-97c7-4a84-8a1f-afe91384caec service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:01.238434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 692/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:18:02.069054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498264-ef60-4aed-a19c-023b4c8c8db9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:02.070089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498264-ef60-4aed-a19c-023b4c8c8db9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:02.070223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9498264-ef60-4aed-a19c-023b4c8c8db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:02.116710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:02.142838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f9498264-ef60-4aed-a19c-023b4c8c8db9 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:02.144083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f9498264-ef60-4aed-a19c-023b4c8c8db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:02.247014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 705/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:02.289947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:02.290504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:02.290603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:02.291944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 693/1398] 10.222.0.47 () {54 vars in 942 bytes} [Tue Jul 12 18:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:18:02.298634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b3fb-8751-44d0-b1d6-71396b071d5a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:02.299020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b3fb-8751-44d0-b1d6-71396b071d5a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:02.299307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b3fb-8751-44d0-b1d6-71396b071d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:02.377609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a246b3fb-8751-44d0-b1d6-71396b071d5a None None] MFA Rules not processed for user `fe3f01883b804652b396cd4e28ab4b46`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:02.378839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a246b3fb-8751-44d0-b1d6-71396b071d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:02.451621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:02.492564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 706/1399] 10.222.0.47 () {58 vars in 1039 bytes} [Tue Jul 12 18:18:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:02.501771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:02.503356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:02.513847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:02.645095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e0e5a0-f543-4269-899a-e2dad8d8738b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:02.753476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:02.760670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:02.761323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:02.761609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:02.761887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:02.763607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:02.856448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:02.858116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb0b22-3f81-4180-918f-f83e00cceec4 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:02.866204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 694/1400] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:18:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:18:03.745431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:03.745431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:03.745431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:03.745431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 707/1401] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:03.750680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11655-482d-4a4b-8ee0-d8067d2fb7bb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:03.751137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11655-482d-4a4b-8ee0-d8067d2fb7bb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:03.751297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f11655-482d-4a4b-8ee0-d8067d2fb7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:03.828439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-06f11655-482d-4a4b-8ee0-d8067d2fb7bb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:03.829704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-06f11655-482d-4a4b-8ee0-d8067d2fb7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:03.920394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:03.940437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 695/1402] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:03.988821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:03.990499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:04.162661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1fee3e-f551-423f-a8e5-5da79cca10f9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:04.212821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:04.298231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:04.298886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:04.299173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:04.299448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:04.301249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:04.398899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:04.400738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337abdc-f149-47ed-962e-82a1113fd5a6 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:04.426821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 708/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:06.766792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:06.767328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:06.767328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:06.772475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 696/1404] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:18:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:06.774878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb932-c5da-41e1-9325-17a4425cbe2f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:06.775288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb932-c5da-41e1-9325-17a4425cbe2f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:06.775449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb932-c5da-41e1-9325-17a4425cbe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:06.849100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:06.861730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d84fb932-c5da-41e1-9325-17a4425cbe2f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:06.863242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d84fb932-c5da-41e1-9325-17a4425cbe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:06.977853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 709/1405] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:18:07.028035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:07.030183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:07.227283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ff901-4ee5-4cfc-80f3-0f27aaac198f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:07.334632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:07.376124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:07.377068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:07.377417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:07.377767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:07.379940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:07.484379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:07.486002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156eec8-27f0-445a-bb62-f7bc288deb3a service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:07.511666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 697/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:18:09.822593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672353-c347-4aa5-9849-c389d444303b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:09.823174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672353-c347-4aa5-9849-c389d444303b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:09.823174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2672353-c347-4aa5-9849-c389d444303b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:09.824478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 710/1407] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:09.829798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c09d-dbe1-402c-8b48-b46fa34f5629 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:09.830264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c09d-dbe1-402c-8b48-b46fa34f5629 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:09.830376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5c09d-dbe1-402c-8b48-b46fa34f5629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:09.904290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-35b5c09d-dbe1-402c-8b48-b46fa34f5629 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:09.905568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-35b5c09d-dbe1-402c-8b48-b46fa34f5629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:09.933851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:10.019688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 698/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:10.068124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672353-c347-4aa5-9849-c389d444303b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:10.070954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b2672353-c347-4aa5-9849-c389d444303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:10.213669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:10.250201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2672353-c347-4aa5-9849-c389d444303b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:10.381513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:10.382453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:10.382736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:10.383011 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:10.384760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:10.495157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:10.499341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:10.501195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8022a9a-4763-415d-bd02-4f900f41819c service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:10.523215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 711/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:20.017684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=33 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:20.533717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:27.095196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18aef9-bb4e-4896-92d8-99429f046511 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:27.095758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18aef9-bb4e-4896-92d8-99429f046511 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:27.095758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18aef9-bb4e-4896-92d8-99429f046511 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:27.097180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 699/1410] 10.222.0.47 () {54 vars in 918 bytes} [Tue Jul 12 18:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:27.103023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b7b5-e82f-4320-b9ed-7453bf2ab9ef None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:27.103448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b7b5-e82f-4320-b9ed-7453bf2ab9ef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:27.103646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d5b7b5-e82f-4320-b9ed-7453bf2ab9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:27.179259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e7d5b7b5-e82f-4320-b9ed-7453bf2ab9ef None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:27.181101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e7d5b7b5-e82f-4320-b9ed-7453bf2ab9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:27.291962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 712/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Jul 12 18:18:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:27.306387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e98a-24da-4f77-b25d-eb7795179684 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:27.306684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e98a-24da-4f77-b25d-eb7795179684 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:27.306970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e98a-24da-4f77-b25d-eb7795179684 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:27.308354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 700/1412] 10.222.0.47 () {54 vars in 918 bytes} [Tue Jul 12 18:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:27.313395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c55c-ac58-4398-b100-abbc8b045df2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:27.313791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c55c-ac58-4398-b100-abbc8b045df2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:27.314171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179c55c-ac58-4398-b100-abbc8b045df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:27.388133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:27.390759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8179c55c-ac58-4398-b100-abbc8b045df2 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:27.391982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8179c55c-ac58-4398-b100-abbc8b045df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:27.509585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 713/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Tue Jul 12 18:18:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:30.017407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:30.017975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:30.017975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:30.019367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:18:30.025462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e950-7c67-4318-aa89-09a12b63205c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:30.025898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e950-7c67-4318-aa89-09a12b63205c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:30.026190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5e950-7c67-4318-aa89-09a12b63205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:30.094234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-79a5e950-7c67-4318-aa89-09a12b63205c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:30.095598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79a5e950-7c67-4318-aa89-09a12b63205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:30.155205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:30.198523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:18:30.242925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:30.244520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:30.394109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94002d-f467-40de-87ae-a4b78994dc65 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:30.467335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:30.509359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:30.510225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:30.510741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:30.511350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:30.513108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:30.601964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:30.603688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf032efa-2fdd-4009-a2ea-7bfaf7933840 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:30.622133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 702/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:31.539944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:31.540552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:31.540552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:31.541902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:31.547294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f38d9-eea2-41f1-a864-6da3bbab0352 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:31.547758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f38d9-eea2-41f1-a864-6da3bbab0352 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:31.547887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f38d9-eea2-41f1-a864-6da3bbab0352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:31.625260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0a5f38d9-eea2-41f1-a864-6da3bbab0352 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:31.626478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0a5f38d9-eea2-41f1-a864-6da3bbab0352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:31.654285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:31.727491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:31.886674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:31.888764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:32.037675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b68897-c7a6-45d3-bfaa-9b6eb75af8c1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:32.081445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:32.158160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:32.158801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:32.159084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:32.159362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:32.161112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:32.251672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:32.253318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c92237-68c1-4e85-ad48-0757ff42b90d service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:32.272376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 716/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:33.163188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:33.163832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:33.163832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:33.165277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:33.171313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156fa6-823b-46d6-bab7-57c629736f53 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:33.178708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156fa6-823b-46d6-bab7-57c629736f53 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:33.178708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42156fa6-823b-46d6-bab7-57c629736f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:33.225666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:33.248615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-42156fa6-823b-46d6-bab7-57c629736f53 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:33.249855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-42156fa6-823b-46d6-bab7-57c629736f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:33.347884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:33.499838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:33.501383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:33.646205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a446c2-e919-40e1-bdf7-4f84339db238 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:33.658120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:33.762719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:33.763387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:33.763671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:33.763947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:33.765640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:33.857353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:33.858977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216324d5-7c9b-4f54-b78e-ef56effe98ef service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:33.877442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 705/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:34.745181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:34.745723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:34.745723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:34.747094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:34.752661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacb42-8af0-4073-99d7-0ec76e59bd48 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:34.753167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacb42-8af0-4073-99d7-0ec76e59bd48 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:34.753283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dacb42-8af0-4073-99d7-0ec76e59bd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:34.771724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:34.827837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-62dacb42-8af0-4073-99d7-0ec76e59bd48 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:34.829088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-62dacb42-8af0-4073-99d7-0ec76e59bd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:34.927622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:35.078803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:35.080276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:35.189761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:35.225988 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee52acf4-762f-45a3-a5b7-ac9e1b2c19cd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:35.340221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:35.340810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:35.341095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:35.341373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:35.343022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:35.420482 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:35.434683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:35.437395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef905df6-1a2e-4833-88b2-80829849a73a service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:35.459806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 719/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:36.342194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:36.342763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:36.342763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:36.344149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:36.349288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de46c91-d29f-4a32-9458-409a92a2d0d1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:36.349772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de46c91-d29f-4a32-9458-409a92a2d0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:36.349885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de46c91-d29f-4a32-9458-409a92a2d0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:36.424908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2de46c91-d29f-4a32-9458-409a92a2d0d1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:36.426175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2de46c91-d29f-4a32-9458-409a92a2d0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:36.524206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:36.675255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:36.676770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:36.761901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:36.824463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2633304-e6bf-4449-bfe6-2df7d978e8d4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:36.936767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:36.937331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:36.937617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:36.937895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:36.939560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:36.988210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:37.027998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:37.029657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff837eb6-6041-4963-a480-bd32fe3e88bb service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:37.049388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 708/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:37.927319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:37.927867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:37.927867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:37.929284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:37.934832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce278-702c-4d27-bcee-77b17454e83e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:37.935294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce278-702c-4d27-bcee-77b17454e83e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:37.935496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ce278-702c-4d27-bcee-77b17454e83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:38.009010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-be8ce278-702c-4d27-bcee-77b17454e83e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:38.010392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-be8ce278-702c-4d27-bcee-77b17454e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:38.110338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:38.261148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:38.262596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:38.312223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:38.410457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204d21e-daa4-46bf-b594-9cea5d445278 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:38.528733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:38.529309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:38.529646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:38.529929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:38.531595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:38.538719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:38.622744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:38.624372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77838d55-3636-4506-98e5-01e154273e85 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:38.642603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 722/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:39.501388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:39.501905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:39.502024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:39.503377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:39.508622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1789b-2227-4c10-a169-ae1f9d22a1b1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:39.509056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1789b-2227-4c10-a169-ae1f9d22a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:39.509185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1789b-2227-4c10-a169-ae1f9d22a1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:39.583899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9dd1789b-2227-4c10-a169-ae1f9d22a1b1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:39.585188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9dd1789b-2227-4c10-a169-ae1f9d22a1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:39.653323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:39.682455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:39.832769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:39.834232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:39.852193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:39.981137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08d4a9-4a74-4d81-907b-992a229faf2c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:40.067352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:40.093739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:40.094403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:40.094689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:40.094969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:40.096734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:40.189468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:40.191104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec18690-4152-4caa-a824-e5f71af6a990 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:40.209549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 711/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:41.068336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:41.068888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:41.068888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:41.070290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:41.075245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf551e4-f3e0-413b-947e-99035a4edb7c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:41.075689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf551e4-f3e0-413b-947e-99035a4edb7c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:41.075840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf551e4-f3e0-413b-947e-99035a4edb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:41.150722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-faf551e4-f3e0-413b-947e-99035a4edb7c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:41.151936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-faf551e4-f3e0-413b-947e-99035a4edb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:41.311954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:41.366221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:41.517114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:41.518719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:41.665702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad86e46-aab3-4215-93d5-e99307a3de81 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:41.720775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:41.781378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:41.782034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:41.782325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:41.782611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:41.784331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:41.875230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:41.876906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079df1f-ac68-4a25-b016-c731367692dc service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:41.895771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 725/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:42.778793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:42.779355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:42.779355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:42.780847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:42.785856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c37a-d8df-42ef-84ed-5182c5697b3a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:42.786330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c37a-d8df-42ef-84ed-5182c5697b3a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:42.786437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c37a-d8df-42ef-84ed-5182c5697b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:42.862090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5e60c37a-d8df-42ef-84ed-5182c5697b3a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:42.863305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5e60c37a-d8df-42ef-84ed-5182c5697b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:42.865380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:42.963162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:43.113424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:43.115622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:43.260839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca994753-9565-43b5-a9ae-a058737ac286 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:43.287333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:43.374619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:43.375274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:43.375567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:43.375847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:43.377536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:43.467801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:43.469435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454cf69-107e-4aeb-93e2-b3b6f6cc1f8c service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:43.487291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 714/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:45.420671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:45.421244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:45.421244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:45.422601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:45.428104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18271-4333-42e3-8890-1a94beb673e6 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:45.428638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18271-4333-42e3-8890-1a94beb673e6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:45.428747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18271-4333-42e3-8890-1a94beb673e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:45.469917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:45.495976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b9d18271-4333-42e3-8890-1a94beb673e6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:45.497228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9d18271-4333-42e3-8890-1a94beb673e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:45.593325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:45.632866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:45.634510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:45.737808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fbc785-ac6b-4f30-a5ba-ab4b3a1506bd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:45.769272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:45.811849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:45.812524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:45.812733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:45.812985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:45.814676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:45.924704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:45.926191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239af490-e50c-4604-a328-9abdf8139434 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:45.944848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 728/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:48.214295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:48.214869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:48.214869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:48.216204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:48.221555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec443a7-1a72-4c9b-a857-684a91457f70 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:48.221991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec443a7-1a72-4c9b-a857-684a91457f70 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:48.222113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec443a7-1a72-4c9b-a857-684a91457f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:48.286289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fec443a7-1a72-4c9b-a857-684a91457f70 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:48.287582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fec443a7-1a72-4c9b-a857-684a91457f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:48.301164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:48.384488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:48.423467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:48.425057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:48.528158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c4f52-b32f-43c8-af64-bba819a5cfa5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:48.540602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:48.601686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:48.602388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:48.603270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:48.603543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:48.605593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:48.712524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:48.714008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e226a43-0622-4ef5-947a-ca3f3da41aab service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:48.732851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 717/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:51.157944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:51.158507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:51.158507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:51.159887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:51.166769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428009fb-a57d-4bff-91c2-7b56c979ce95 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:51.167226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428009fb-a57d-4bff-91c2-7b56c979ce95 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:51.167365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428009fb-a57d-4bff-91c2-7b56c979ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:51.215286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:51.234699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-428009fb-a57d-4bff-91c2-7b56c979ce95 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:51.235921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-428009fb-a57d-4bff-91c2-7b56c979ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:51.332367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:51.372288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:51.373928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:51.484109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b597c-e500-46bd-8e7b-52106ebac6d4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:51.527017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:51.561951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:51.562630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:51.562917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:51.563196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:51.564896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:51.704177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:51.705579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71522f7-087d-4e62-8bed-5a7996c920c7 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:51.724409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 731/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:53.895746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:53.896277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:53.896388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:53.897774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:53.903411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eebd68-fdba-4b58-a360-0f49efb29822 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:53.903840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eebd68-fdba-4b58-a360-0f49efb29822 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:53.903982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eebd68-fdba-4b58-a360-0f49efb29822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:53.969631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d0eebd68-fdba-4b58-a360-0f49efb29822 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:53.970850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d0eebd68-fdba-4b58-a360-0f49efb29822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:53.992011 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:54.071792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:54.111544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:54.113165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:54.219206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2106334-d59c-4f4d-9251-efd42079a69d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:54.239365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:54.294454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:54.295114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:54.295402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:54.295684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:54.297442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:54.406918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:54.408390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce26f8e-9a2a-4af5-85ea-cb869dd0b173 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:54.426316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 720/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:56.686304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:56.686849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:56.687324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:56.689697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:56.695236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3fb8-3c98-45ba-b72f-35272b808cd7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:56.695712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3fb8-3c98-45ba-b72f-35272b808cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:56.695870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c3fb8-3c98-45ba-b72f-35272b808cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:56.763143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:56.766309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3b8c3fb8-3c98-45ba-b72f-35272b808cd7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:56.767557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3b8c3fb8-3c98-45ba-b72f-35272b808cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:56.870759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:56.909188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:56.910762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:57.026323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d79d4-9341-41ae-87d2-90364b6af731 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:57.078441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:57.105501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:57.106156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:57.106440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:57.106715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:57.108459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:57.237104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:57.238582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a56ab-2bd4-4fba-a70c-f9ddcc7f4ac2 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:57.259104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 734/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:18:59.377565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:59.378111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:59.378234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:59.379506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:18:59.384949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e88f0-2509-455a-afd5-612e3432b679 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:59.385473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e88f0-2509-455a-afd5-612e3432b679 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:59.385564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e88f0-2509-455a-afd5-612e3432b679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:59.454782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e24e88f0-2509-455a-afd5-612e3432b679 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:18:59.456033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e24e88f0-2509-455a-afd5-612e3432b679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:59.485111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:59.552150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:18:59.590509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:18:59.592057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:59.698364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b9190-55d4-4d7d-8ab7-ef280f28b556 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:18:59.726482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:18:59.775232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:18:59.775896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:18:59.776184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:18:59.776501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:18:59.778235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:18:59.888415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:18:59.890222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff512abc-c4b5-4fa2-8268-8d73275c352b service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:18:59.909098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 723/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:02.028628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:02.029207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:02.029207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:02.030576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:02.035956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718351-537d-4345-8151-ce9000173014 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:02.036414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718351-537d-4345-8151-ce9000173014 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:02.036590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718351-537d-4345-8151-ce9000173014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:02.104610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a4718351-537d-4345-8151-ce9000173014 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:02.106106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a4718351-537d-4345-8151-ce9000173014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:02.114802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:02.207158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:02.245231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:02.246928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:02.351400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd641a-6acc-41d6-9c13-289f838f88c1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:02.409079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:02.427570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:02.428234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:02.428601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:02.428889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:02.431120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:02.546863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:02.548229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e91424-adc3-495d-b896-cf50a74c362a service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:02.568907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 737/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:04.672701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:04.673310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:04.673310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:04.674659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:04.679953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd66-291e-491b-9c60-6f0f5552e2b5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:04.681008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd66-291e-491b-9c60-6f0f5552e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:04.681265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385bd66-291e-491b-9c60-6f0f5552e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:04.747836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4385bd66-291e-491b-9c60-6f0f5552e2b5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:04.749114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4385bd66-291e-491b-9c60-6f0f5552e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:04.788144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:04.849199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:04.887658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:04.889308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:05.000036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e417a38-520c-45ab-87e2-4ad7c779b620 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:05.036859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:05.074209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:05.074860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:05.075140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:05.075412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:05.077199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:05.185215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:05.186672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6885df-c0c9-4668-bee9-158470b02969 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:05.205530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 726/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:07.319050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:07.319608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:07.319608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:07.321577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:07.327041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810850-a22b-457a-9303-de917aa069cd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:07.327460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810850-a22b-457a-9303-de917aa069cd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:07.327638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36810850-a22b-457a-9303-de917aa069cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:07.393420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-36810850-a22b-457a-9303-de917aa069cd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:07.394651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-36810850-a22b-457a-9303-de917aa069cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:07.412556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:07.494492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:07.531108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:07.532730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:07.639826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee63aa-c1b2-4e20-bfbc-30d0f2d4bafe None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:07.708807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:07.715755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:07.721057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:07.721057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:07.721057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:07.721057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:07.832675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:07.834053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddba18-438e-43ef-b815-8a5e33806511 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:07.853011 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 740/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:09.919014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:09.919569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:09.919569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:09.920981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:09.926131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98309aa7-f11b-4c24-a307-953365a4992f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:09.926545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98309aa7-f11b-4c24-a307-953365a4992f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:09.926702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98309aa7-f11b-4c24-a307-953365a4992f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:09.992343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-98309aa7-f11b-4c24-a307-953365a4992f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:09.994095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-98309aa7-f11b-4c24-a307-953365a4992f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:10.041083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:10.089972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:10.126954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:10.128530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:10.233331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839417f7-286e-46df-afe5-399ca09d26a6 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:10.279724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:10.307471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:10.308126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:10.308445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:10.308726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:10.310397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:10.416850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:10.418311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994b60d-c7ed-43d9-93ef-6c780c6f37ae service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:10.436939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 729/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:20.088137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=22 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:20.434642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=66 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:30.493080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec70f6-af9f-42bb-9324-a988c9a02f03 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:30.493621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec70f6-af9f-42bb-9324-a988c9a02f03 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:30.493621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec70f6-af9f-42bb-9324-a988c9a02f03 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:30.495046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 742/1471] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:30.500563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7e65-ca97-4574-b75c-7e37d3d97803 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:30.501063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7e65-ca97-4574-b75c-7e37d3d97803 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:30.501159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb7e65-ca97-4574-b75c-7e37d3d97803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:30.569570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-becb7e65-ca97-4574-b75c-7e37d3d97803 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:30.571431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-becb7e65-ca97-4574-b75c-7e37d3d97803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:30.667662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 730/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:30.754497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:30.754795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:30.755081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:30.756478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 743/1473] 10.222.0.47 () {54 vars in 962 bytes} [Tue Jul 12 18:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:30.762238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66119af-e209-46f0-977c-2196e5a5850b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:30.762996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66119af-e209-46f0-977c-2196e5a5850b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:30.763285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66119af-e209-46f0-977c-2196e5a5850b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:30.839130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c66119af-e209-46f0-977c-2196e5a5850b None None] MFA Rules not processed for user `b32ef598390145d1b88c3e3811edb79b`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:30.840395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c66119af-e209-46f0-977c-2196e5a5850b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:30.862730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:30.919547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 731/1474] 10.222.0.47 () {58 vars in 1059 bytes} [Tue Jul 12 18:19:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:30.927209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:30.928790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:31.036121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728eb5e-d75b-4645-b0b8-ffdba25a49e3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:31.112457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:31.113123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:31.113419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:31.113730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:31.115431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:31.170938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:31.228353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:31.229814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d46e9-e094-4983-b4b2-c60c349f8bd3 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:31.250605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 744/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:33.382420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:33.382994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:33.383268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:33.384739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 732/1476] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:33.389927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09536f-c8bc-4ac7-aea1-708f64b87c1a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:33.390370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09536f-c8bc-4ac7-aea1-708f64b87c1a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:33.390480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09536f-c8bc-4ac7-aea1-708f64b87c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:33.455667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2a09536f-c8bc-4ac7-aea1-708f64b87c1a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:33.456928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2a09536f-c8bc-4ac7-aea1-708f64b87c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:33.553930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 745/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:33.577783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:33.579328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:33.704335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb74dd-b05d-4815-b665-28e0ee9eb67d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:33.754815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:33.802727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:33.803559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:33.803842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:33.804243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:33.805062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:33.806919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:19:33.806919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:19:33.806919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:19:33.808064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea29e71-b7c6-47fe-bb80-54ceb8538f4d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:33.827616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 733/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Tue Jul 12 18:19:33 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:19:35.896137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:35.896793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:35.896793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:35.898604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 746/1479] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:35.903956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f804-6579-48a8-9a1f-4aa1ac5e9272 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:35.904417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f804-6579-48a8-9a1f-4aa1ac5e9272 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:35.904558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f804-6579-48a8-9a1f-4aa1ac5e9272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:35.968935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4bc5f804-6579-48a8-9a1f-4aa1ac5e9272 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:35.970071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4bc5f804-6579-48a8-9a1f-4aa1ac5e9272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:36.066591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 734/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:36.111798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:36.111798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:36.122543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:36.261158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0dd54-fa29-43c8-960c-53dc494fc6a9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:36.342254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:36.376762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:36.377412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:36.377698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:36.377977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:36.379621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:36.470028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:36.471637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777723a0-ed3f-4596-b4b5-a41a193080c5 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:36.490431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 747/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:38.714844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:38.715395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:38.715395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:38.716710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 735/1482] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:38.722006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113379-5c07-49e7-9317-195de5201a6a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:38.722434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113379-5c07-49e7-9317-195de5201a6a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:38.722567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113379-5c07-49e7-9317-195de5201a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:38.791303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-20113379-5c07-49e7-9317-195de5201a6a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:38.792615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-20113379-5c07-49e7-9317-195de5201a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:38.840393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:38.891891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 748/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:38.928735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:38.930716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:38.957202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:39.044029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fd18a-087a-4648-a44c-74bc3e907538 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:39.119873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:39.120562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:39.120848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:39.121123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:39.122863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:39.199450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:39.237933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:39.239494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1b356-a391-4d15-9ecf-254ed6252381 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:39.263802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 736/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:41.345975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:41.346534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:41.346534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:41.347853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 749/1485] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:41.353950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ad8-ad06-450a-863b-a501b2a0cbef None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:41.354506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ad8-ad06-450a-863b-a501b2a0cbef None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:41.354506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ad8-ad06-450a-863b-a501b2a0cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:41.425804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-20006ad8-ad06-450a-863b-a501b2a0cbef None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:41.427035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-20006ad8-ad06-450a-863b-a501b2a0cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:41.531359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 737/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:19:41.571363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:41.572997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:41.678151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b0787-a896-4c6e-a6be-52255f304882 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:41.750478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:41.752793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:41.753452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:41.753744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:41.754028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:41.755746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:41.869719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:41.871169 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4c61-9d83-456b-bce7-ec648f2076e4 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:41.889763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 750/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:44.033044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:44.033618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:44.033618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:44.034982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 738/1488] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:19:44.040864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bde1-a032-47ea-bc66-4f246d80e473 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:44.041294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bde1-a032-47ea-bc66-4f246d80e473 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:44.041429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575bde1-a032-47ea-bc66-4f246d80e473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:44.107599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7575bde1-a032-47ea-bc66-4f246d80e473 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:44.108848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7575bde1-a032-47ea-bc66-4f246d80e473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:44.165026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:44.210685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 751/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:44.251145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:44.252837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:44.295818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:44.358236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ab552-d38e-4584-b55e-a0786c6b0b05 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:44.432606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:44.433265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:44.433554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:44.433836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:44.435499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:44.516266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:44.545714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:44.547233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15d92d-443a-40f0-ab24-5790a2863136 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:44.565442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 739/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:46.942082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:46.942669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:46.942669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:46.944041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 752/1491] 10.222.0.47 () {54 vars in 926 bytes} [Tue Jul 12 18:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:19:46.950318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5cad8-9baa-499d-a794-e70b99f37e32 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:46.950852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5cad8-9baa-499d-a794-e70b99f37e32 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:46.950948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5cad8-9baa-499d-a794-e70b99f37e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:47.022258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dbb5cad8-9baa-499d-a794-e70b99f37e32 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:19:47.024107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dbb5cad8-9baa-499d-a794-e70b99f37e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:47.125661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 740/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Tue Jul 12 18:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:19:47.165168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:19:47.166720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:47.281633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98a941-4643-4ffb-9b7f-fd0fa251ef2a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:19:47.361018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:19:47.361677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:19:47.361969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:19:47.362256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:19:47.363956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:19:47.374315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:47.475284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:19:47.477600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f369bb3-2c44-448c-b2f4-66a0107a6d07 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:19:47.496670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 753/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:19:57.123801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=82 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:19:57.494360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=50 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:24.009985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:24.010594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:24.010594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:24.089699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:21:24.090945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:24.191745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 741/1494] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:21:24.200075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:24.200653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:24.200653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:24.279764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:21:24.281089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:24.383386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 754/1495] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:21:24.388250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:24.389497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:24.400747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ddcb7-d8c4-4419-a691-d74a28d79b2c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:24.463212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:24.497819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:24.498487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:24.498833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:24.499129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:24.499868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:21:24.500175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:24.501254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:24.501254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:21:24.501254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:21:24.502143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:24.515640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 742/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:21:24.524907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:24.536199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ae605-af6f-4230-a64f-b80861f1ac4c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:24.605881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:24.642522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:24.643226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:24.643513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:24.643792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:24.645287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:24.646902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:24.646902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:21:24.646902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:21:24.648112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:24.789995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 755/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:21:24 2022] POST /identity/v3/projects => generated 379 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:21:24.793961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:24.795494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:24.833147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5891d4-2bb5-4903-8f30-1b0ce552f585 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:24.933823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:24.934498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:24.934778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:24.935053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:24.935754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:24.936813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:24.936813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:21:24.936813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:21:24.938308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:25.031472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 743/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:21:24 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:21:25.035958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:25.041480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:25.101462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40c1d7-2c84-49ab-afdb-650b399e580e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:25.195508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:25.210914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:25.213085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.213085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.213085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.213085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:25.213943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:25.213943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:21:25.213943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:21:25.214843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:25.228095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 756/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:21:25.232479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:25.234012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:25.245618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b08788-cb67-4c8f-8ba1-f77f9d317d9c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:25.271643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:25.347280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:25.347927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.348300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.348612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.349354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:25.350399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:25.350399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:21:25.350399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:21:25.351302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f24d0-7cbc-4304-8e8b-68d726a3e6ee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:25.365413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 744/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:21:25.369878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:25.371492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:25.409875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8480-59d1-428a-a230-9b22dac6fe67 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:25.518430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:25.519336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.519629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.520691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] PATH_INFO: `/v3/projects/7655b3a4dc8a4e2e9c5565a3d3f54b77/users/a101877611914f7d896abc2ea318ab96/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.545199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7655b3a4dc8a4e2e9c5565a3d3f54b77, user_id=a101877611914f7d896abc2ea318ab96, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:25.546267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:25.546267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:21:25.546267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:21:25.548356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1a351-30a3-4ebe-9f61-03f61e0571b1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:25.639550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 757/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:21:25 2022] PUT /identity/v3/projects/7655b3a4dc8a4e2e9c5565a3d3f54b77/users/a101877611914f7d896abc2ea318ab96/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:21:25.652260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957f0f-3273-42e0-9f98-9726a3841fce None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.652590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957f0f-3273-42e0-9f98-9726a3841fce None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.652876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58957f0f-3273-42e0-9f98-9726a3841fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.717407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-58957f0f-3273-42e0-9f98-9726a3841fce None None] MFA Rules not processed for user `a101877611914f7d896abc2ea318ab96`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:21:25.718686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58957f0f-3273-42e0-9f98-9726a3841fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:25.805290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 745/1502] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:21:25.819857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.819857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.820076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.821351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 758/1503] 10.222.0.47 () {54 vars in 963 bytes} [Tue Jul 12 18:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:21:25.827405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea7ad-e3ff-4204-8e7f-fc2a5e93a8c5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:25.827705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea7ad-e3ff-4204-8e7f-fc2a5e93a8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:25.828132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdea7ad-e3ff-4204-8e7f-fc2a5e93a8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:25.864053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:25.912141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ebdea7ad-e3ff-4204-8e7f-fc2a5e93a8c5 None None] MFA Rules not processed for user `44fe9acbbbdc463ab22869886df4e3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:21:25.913316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ebdea7ad-e3ff-4204-8e7f-fc2a5e93a8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:25.993996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 746/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Tue Jul 12 18:21:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:21:26.001767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:26.003320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:26.057136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:26.115349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d176929-62d6-4244-907d-669709a45ea1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:26.190340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:26.190911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:26.191204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:26.191489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:26.193383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:26.292613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:26.294074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:26.297626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:26.313252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 759/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:21:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:21:26.356099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7af95-471a-415e-8166-4eec7524d487 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:26.356572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7af95-471a-415e-8166-4eec7524d487 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:26.356699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7af95-471a-415e-8166-4eec7524d487 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:26.358053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 747/1506] 10.222.0.47 () {52 vars in 829 bytes} [Tue Jul 12 18:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:21:26.361674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:26.362926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:26.463545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808118b-8f11-4561-8680-6f19fa07b43a service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:26.560358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:26.561236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:26.561524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:26.561801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] PATH_INFO: `/v3/users/a101877611914f7d896abc2ea318ab96` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:26.574090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a101877611914f7d896abc2ea318ab96)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:26.575135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:26.575135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:21:26.575135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:21:26.576394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:26.587805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:26.777442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a101877611914f7d896abc2ea318ab96'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:26.793344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a101877611914f7d896abc2ea318ab96'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:26.809493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a101877611914f7d896abc2ea318ab96'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:26.810083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a101877611914f7d896abc2ea318ab96'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:26.810510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a101877611914f7d896abc2ea318ab96'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:26.899275 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 760/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:21:26 2022] DELETE /identity/v3/users/a101877611914f7d896abc2ea318ab96 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:21:26.903673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:26.903968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:26.904250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:26.959799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:21:26.961046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:27.037868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:27.050998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 748/1508] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:21:27.060811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:27.062249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:27.167257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cdc5c-eaf7-4656-ac88-783048d6f121 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:27.224515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:27.244550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:27.245205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:27.245492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:27.245772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:27.247430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:27.378113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:27.380428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c700a16-6fc2-4416-b670-9e13c864d841 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:27.402791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 761/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:21:27.857801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:21:27.859076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:21:27.871319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9603-7d60-4b18-a978-f430b8d53a2b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:21:27.971535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:21:27.972433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:21:27.972740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:21:27.973569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] PATH_INFO: `/v3/projects/7655b3a4dc8a4e2e9c5565a3d3f54b77` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:21:27.984167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7655b3a4dc8a4e2e9c5565a3d3f54b77)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:21:27.985277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:21:27.985277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:21:27.985277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:21:27.986482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:21:28.094398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7655b3a4dc8a4e2e9c5565a3d3f54b77', 'request_id': 'req-a9b116bf-0834-4e22-a649-b4744e9f971b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:28.108500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7655b3a4dc8a4e2e9c5565a3d3f54b77', 'request_id': 'req-a9b116bf-0834-4e22-a649-b4744e9f971b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:28.176389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7655b3a4dc8a4e2e9c5565a3d3f54b77', 'request_id': 'req-a9b116bf-0834-4e22-a649-b4744e9f971b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:28.199964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] The token cache is being invalidate because project 7655b3a4dc8a4e2e9c5565a3d3f54b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:21:28.200259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:28.200726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a9b116bf-0834-4e22-a649-b4744e9f971b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:21:28.202191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 749/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:21:27 2022] DELETE /identity/v3/projects/7655b3a4dc8a4e2e9c5565a3d3f54b77 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:21:37.649859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=62 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:21:38.197952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:23:07.349572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:07.350121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:07.350230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:07.351580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:07.356765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52427d-e77e-4838-9ca0-d5fcb9d600f2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:07.357233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52427d-e77e-4838-9ca0-d5fcb9d600f2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:07.357346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52427d-e77e-4838-9ca0-d5fcb9d600f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:07.434878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8c52427d-e77e-4838-9ca0-d5fcb9d600f2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:07.436090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8c52427d-e77e-4838-9ca0-d5fcb9d600f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:07.544196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:23:07.698177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:07.700298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:07.821921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a6de8-3c32-442d-a0f3-29a3d13cd9e5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:07.827385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:07.828028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:07.828332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:07.828619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:07.837287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:07.838876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238c1f-75e7-4559-aefc-47a28f226080 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:07.848931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 763/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:08.677831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263908-743d-4b5b-a5c8-7f5a43931529 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:08.678399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263908-743d-4b5b-a5c8-7f5a43931529 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:08.678399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263908-743d-4b5b-a5c8-7f5a43931529 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:08.679763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:08.684831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b9a2c-fcc1-4163-9a81-f7db7c808fd8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:08.685268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b9a2c-fcc1-4163-9a81-f7db7c808fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:08.685399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b9a2c-fcc1-4163-9a81-f7db7c808fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:08.703826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6a6b9a2c-fcc1-4163-9a81-f7db7c808fd8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:08.705527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6a6b9a2c-fcc1-4163-9a81-f7db7c808fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:08.713521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:08.866023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e263908-743d-4b5b-a5c8-7f5a43931529 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:08.880009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e263908-743d-4b5b-a5c8-7f5a43931529 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:08.885465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:08.886181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:08.886466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:08.886744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:08.895661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:08.897318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec255bd-8fea-49f8-b6d2-d4005432fc42 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:08.906745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 752/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:09.047930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07567ac-d9cc-4c01-a70c-bd78d5ca22ef None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:09.048225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07567ac-d9cc-4c01-a70c-bd78d5ca22ef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:09.048540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07567ac-d9cc-4c01-a70c-bd78d5ca22ef None None] PATH_INFO: `/v3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:09.049921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 765/1517] 10.222.0.26 () {54 vars in 925 bytes} [Tue Jul 12 18:23:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:23:09.057537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dfa96-9ed9-487e-b4ce-589c3d71e8c8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:09.057838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dfa96-9ed9-487e-b4ce-589c3d71e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:09.058122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dfa96-9ed9-487e-b4ce-589c3d71e8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:09.119495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-933dfa96-9ed9-487e-b4ce-589c3d71e8c8 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:09.120750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-933dfa96-9ed9-487e-b4ce-589c3d71e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:09.137857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:23:09.192894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 753/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Jul 12 18:23:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:18.215208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c886a3-d3b0-47c7-a608-7ccd1850942e None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:18.215779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c886a3-d3b0-47c7-a608-7ccd1850942e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:18.215779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c886a3-d3b0-47c7-a608-7ccd1850942e None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:18.217151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 766/1519] 10.222.0.47 () {54 vars in 960 bytes} [Tue Jul 12 18:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:18.222702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301da4cb-41ec-4065-9495-b06bf5d541d7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:18.223219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301da4cb-41ec-4065-9495-b06bf5d541d7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:18.223338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301da4cb-41ec-4065-9495-b06bf5d541d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:18.242019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-301da4cb-41ec-4065-9495-b06bf5d541d7 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:18.243240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-301da4cb-41ec-4065-9495-b06bf5d541d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:18.252497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 754/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Tue Jul 12 18:23:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:18.260150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c886a3-d3b0-47c7-a608-7ccd1850942e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:18.274079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c886a3-d3b0-47c7-a608-7ccd1850942e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:18.279191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:18.279849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:18.280137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:18.280453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:18.289927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:18.291589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316b7de-0b66-448a-8a26-8d5f72320170 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:18.303274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 767/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:20.191293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaa1-4ca8-413e-946a-ea05490cb41b None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:20.191861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaa1-4ca8-413e-946a-ea05490cb41b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:20.191861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abeaa1-4ca8-413e-946a-ea05490cb41b None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:20.193289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:20.198281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a673aa-1e9b-4035-8285-0347ef90b7a5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:20.198730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a673aa-1e9b-4035-8285-0347ef90b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:20.198929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a673aa-1e9b-4035-8285-0347ef90b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:20.217570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-77a673aa-1e9b-4035-8285-0347ef90b7a5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:20.218788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-77a673aa-1e9b-4035-8285-0347ef90b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:20.226758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:20.380969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abeaa1-4ca8-413e-946a-ea05490cb41b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:20.396196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abeaa1-4ca8-413e-946a-ea05490cb41b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:20.401650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:20.402286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:20.402571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:20.402849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:20.411456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:20.413089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0960f27-a648-42f8-8541-0b5655409b0d service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:20.422555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 756/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:21.253274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00852-91cb-4f97-8393-2dc3ae39c93d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:21.253827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00852-91cb-4f97-8393-2dc3ae39c93d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:21.253827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa00852-91cb-4f97-8393-2dc3ae39c93d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:21.255279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 18:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:21.260197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a68339-8396-403e-a753-6aee8583001c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:21.260670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a68339-8396-403e-a753-6aee8583001c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:21.260786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a68339-8396-403e-a753-6aee8583001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:21.279434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e4a68339-8396-403e-a753-6aee8583001c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:21.280704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4a68339-8396-403e-a753-6aee8583001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:21.290784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 18:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:21.445680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa00852-91cb-4f97-8393-2dc3ae39c93d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:21.461030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa00852-91cb-4f97-8393-2dc3ae39c93d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:21.466530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:21.467172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:21.467460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:21.467739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:21.476848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:21.478430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ef4bd-fc9c-499e-a393-e516f5f56175 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:21.488221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 770/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:25.492860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:25.493449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:25.493564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:25.494951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:25.501081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc620de1-16fd-4394-ac4e-1963b01ae666 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:25.504472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc620de1-16fd-4394-ac4e-1963b01ae666 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:25.504472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc620de1-16fd-4394-ac4e-1963b01ae666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:25.528759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bc620de1-16fd-4394-ac4e-1963b01ae666 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:25.530278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bc620de1-16fd-4394-ac4e-1963b01ae666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:25.541422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:25.628694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:25.630573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:25.731698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0a8e3-9075-440a-8e5e-0de516141528 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:25.735139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:25.735815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:25.736098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:25.736511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:25.749654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:25.752040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a925aba-8885-4de1-91da-aac35d119505 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:25.763057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 759/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:27.867023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:27.867559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:27.867559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:27.869004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:27.874168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb8f3-b6b6-4f02-a73a-5c8e189af57e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:27.874631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb8f3-b6b6-4f02-a73a-5c8e189af57e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:27.874738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb8f3-b6b6-4f02-a73a-5c8e189af57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:27.894024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e0ddb8f3-b6b6-4f02-a73a-5c8e189af57e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:27.895228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e0ddb8f3-b6b6-4f02-a73a-5c8e189af57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:27.903339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:27.940921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:27.942674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:27.950357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=75 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:23:27.956849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae908d-4d7e-45a7-8b3a-b0d6e3d1818b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:27.960515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:27.971024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:27.971024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:27.971024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:27.973352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:27.974802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041c651-673d-4ca7-b809-276bb34d9f8c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:27.984815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 773/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:31.543611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc973-4b16-454e-a524-69e9710123db None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:31.544173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc973-4b16-454e-a524-69e9710123db None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:31.544173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bc973-4b16-454e-a524-69e9710123db None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:31.545580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:31.551565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d1ee-3a5c-44cd-9615-3c9683199807 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:31.552080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d1ee-3a5c-44cd-9615-3c9683199807 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:31.552239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d1ee-3a5c-44cd-9615-3c9683199807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:31.570518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5ac5d1ee-3a5c-44cd-9615-3c9683199807 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:31.571770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5ac5d1ee-3a5c-44cd-9615-3c9683199807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:31.579704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:23:31.887853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bc973-4b16-454e-a524-69e9710123db None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:31.889557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-608bc973-4b16-454e-a524-69e9710123db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:31.955945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:23:31.969031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bc973-4b16-454e-a524-69e9710123db None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:31.972812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:31.973471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:31.973757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:31.974045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:31.983464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:31.984946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903eef8-0b49-4f6c-a09d-6769149e3fb4 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:31.995523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 762/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:34.042909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:34.043462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:34.043462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:34.044827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 18:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:23:34.050576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581623-b9d1-4899-8e52-df7b3cd5958c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:34.051015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581623-b9d1-4899-8e52-df7b3cd5958c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:34.051151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581623-b9d1-4899-8e52-df7b3cd5958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:34.068870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b4581623-b9d1-4899-8e52-df7b3cd5958c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:23:34.070062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b4581623-b9d1-4899-8e52-df7b3cd5958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:34.077833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 18:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:23:34.378589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:23:34.380723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:23:34.394761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eacd0c-0638-4b27-842c-a367fe6c694d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:23:34.398299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:23:34.398941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:23:34.399229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:23:34.399530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:23:34.409166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:23:34.410610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db93fd1-ceb7-4827-a7d9-4716a80ce1e2 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:23:34.420230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 776/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:23:44.063653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=11 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:23:44.415956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=14 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:02.515043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:02.516491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:02.516491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:02.549300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:02.550767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:02.567355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 764/1540] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:02.577532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:02.578139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:02.578139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:02.614926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:02.616227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:02.633941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 777/1541] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:02.640248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:02.664339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaea2d3-c858-4270-b352-7f606e878a96 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:02.664893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:02.665605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:02.671961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:02.671961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:02.671961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:02.671961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:02.673711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:02.708861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 765/1542] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:02.718199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:02.730205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9beb24-54e8-4b09-b0f0-dac3aface8e6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:02.745360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:02.751397 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:02.752127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:02.752778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:02.754019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:02.756592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e57bb-9b8e-46a9-af79-aced6cabbf3a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:02.967316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 778/1543] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:02 2022] POST /identity/v3/projects => generated 393 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:02.985466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.000232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a993e4-4ce8-4b5d-bb55-13d2f2157e20 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.095901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccbc9-a0f5-4a1a-8b68-17beeaa606b4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.096273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccbc9-a0f5-4a1a-8b68-17beeaa606b4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.097062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccbc9-a0f5-4a1a-8b68-17beeaa606b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.131536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1f1ccbc9-a0f5-4a1a-8b68-17beeaa606b4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.137664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1f1ccbc9-a0f5-4a1a-8b68-17beeaa606b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.185873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.186579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.186866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.187187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.187893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.190010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813f83b-19e5-4685-8eea-b036ad876de6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.237204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 779/1544] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:03.238594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911b79f-6f78-4a52-ae3b-b617ba4bdfb0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.238890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911b79f-6f78-4a52-ae3b-b617ba4bdfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.239174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911b79f-6f78-4a52-ae3b-b617ba4bdfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.271509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4911b79f-6f78-4a52-ae3b-b617ba4bdfb0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.271509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4911b79f-6f78-4a52-ae3b-b617ba4bdfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.282771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 780/1545] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.284960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b55d0-84e4-437b-8737-ffc5ae016338 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.284960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b55d0-84e4-437b-8737-ffc5ae016338 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.284960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b55d0-84e4-437b-8737-ffc5ae016338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.304811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b53b55d0-84e4-437b-8737-ffc5ae016338 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.306089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b53b55d0-84e4-437b-8737-ffc5ae016338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.314517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 766/1546] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:02 2022] POST /identity/v3/users => generated 297 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:03.315632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 781/1547] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.315766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a389-1e66-47b1-8fd8-6cbaec4e5c66 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.316024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a389-1e66-47b1-8fd8-6cbaec4e5c66 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.316341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70a389-1e66-47b1-8fd8-6cbaec4e5c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.316732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ad969-c731-47fd-9ba8-785fb6f1fde2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.317029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ad969-c731-47fd-9ba8-785fb6f1fde2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.317306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ad969-c731-47fd-9ba8-785fb6f1fde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.338041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0e70a389-1e66-47b1-8fd8-6cbaec4e5c66 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.339279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0e70a389-1e66-47b1-8fd8-6cbaec4e5c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.342677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b65ad969-c731-47fd-9ba8-785fb6f1fde2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.343876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b65ad969-c731-47fd-9ba8-785fb6f1fde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.347482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 767/1548] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.348839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.349135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.349424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.353207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 782/1549] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.354788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625beb-d976-4273-bbbb-5a1778c03c49 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.355085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625beb-d976-4273-bbbb-5a1778c03c49 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.355366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1625beb-d976-4273-bbbb-5a1778c03c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.374230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d1625beb-d976-4273-bbbb-5a1778c03c49 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.375084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.375465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d1625beb-d976-4273-bbbb-5a1778c03c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.376261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.384564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 783/1550] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.385297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 768/1551] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.386123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.389424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990061d2-9fd6-4b2a-9941-21294e20b64a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.390047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990061d2-9fd6-4b2a-9941-21294e20b64a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.390565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990061d2-9fd6-4b2a-9941-21294e20b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.396538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b5a8b-db05-425c-8bcc-b32ff57356fd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.402275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.403014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.405099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.405099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.405099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.406173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6450d-eb40-4cca-a790-2fe3c04f4a0c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.412915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-990061d2-9fd6-4b2a-9941-21294e20b64a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.414455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-990061d2-9fd6-4b2a-9941-21294e20b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.420089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 769/1552] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:03.420089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.420089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.420089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.426290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 784/1553] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.427954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762379d9-53fd-48b6-9112-c9b3916194bc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.428113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762379d9-53fd-48b6-9112-c9b3916194bc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.428340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762379d9-53fd-48b6-9112-c9b3916194bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.440592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.441977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.448463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-762379d9-53fd-48b6-9112-c9b3916194bc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.449730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-762379d9-53fd-48b6-9112-c9b3916194bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.451737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 770/1554] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.452701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.465580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 785/1555] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.465802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb03bf9-01b5-4602-8ac9-796d3fa8e4e0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.467009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984807-b1ef-4d36-b4d6-8d82a90ffcc3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.467303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984807-b1ef-4d36-b4d6-8d82a90ffcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.467584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984807-b1ef-4d36-b4d6-8d82a90ffcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.472993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.479921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0cb151-c941-48bd-be95-223b61472f7f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.486880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-06984807-b1ef-4d36-b4d6-8d82a90ffcc3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.488261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-06984807-b1ef-4d36-b4d6-8d82a90ffcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.491196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 771/1556] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.492768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.493064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.493381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.503083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 786/1557] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.504489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.504856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.505198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.522041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.523883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.527307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.528520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.531924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 772/1558] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.533573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.536663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 787/1559] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.537469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.545764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466bb33-a68d-4d5a-b328-ac868cbdf2b8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.548532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3b7ca-495a-426a-8f5d-74cb424888fd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.550413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.551085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.551462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.551572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.551673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.552195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.552452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.552552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.552778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.553546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.553856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.554132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ae44a-7643-4682-aa5d-b63291064994 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.554928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:03.554928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:25:03.554928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:25:03.555940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.565180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 773/1560] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:03.566690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.567059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.567415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.567804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 788/1561] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.575956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.587925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01897375-0129-4503-aa26-cc9e5b05eb1f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.593382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.594055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.594345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.594631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.595373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.595482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:03.595683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.596741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:03.597339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.605963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 774/1562] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:03.606788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.616641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 789/1563] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.616641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.622659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a202d-56fe-478b-a95d-8c3a07d60e7f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.623033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e5073-30a2-4213-a3e0-85a991a95e33 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.627325 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.627539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.627996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.628191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.628319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.628510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.628632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.628795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.629309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.629528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.629844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.630402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:03.630402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:25:03.630402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:25:03.631441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.631646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.644212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 790/1564] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.645030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.657546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3dbc5-42e5-47ab-8337-b75503e1c5ad admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.662501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.667854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.679611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 791/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.680643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.691245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772c712-6998-41c8-890c-8deefa8dad2c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.696741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.697487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.697773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.698053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.698804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:03.699114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.700910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.714056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 792/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:03.715639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.726758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:03.734686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888693d-3723-4be9-97ea-87e4ba0fc02d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.738672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 775/1567] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:03.738672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.783163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:03.803366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.803632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468219e-59cc-4dff-a657-550eb66e7699 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:03.804842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.805128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.805416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] PATH_INFO: `/v3/projects/54167aa5315d4dd2b514ad4b027634f9/users/b58b0d520aa1418bb6ad3b5645a826a7/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.809473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:03.810152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:03.810518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:03.813567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:03.813567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.813567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.824759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54167aa5315d4dd2b514ad4b027634f9, user_id=b58b0d520aa1418bb6ad3b5645a826a7, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:03.825073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:03.931983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 793/1568] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:03 2022] PUT /identity/v3/projects/54167aa5315d4dd2b514ad4b027634f9/users/b58b0d520aa1418bb6ad3b5645a826a7/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:03.932892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:03.996282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d24c92-1095-4a54-a4a0-04427fa62cb4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.001204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.001880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.002168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.002452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.003188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:04.003608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.005185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.017550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 776/1569] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] POST /identity/v3/projects => generated 383 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.017550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.025816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 794/1570] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:04.025816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.030153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5abb17-9b09-4a75-a2ea-67b50a1b4f85 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.092581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e03d22-1c87-44ec-ad81-9707a3746863 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.093508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.094185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.094483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.094768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.095478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.097448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.098041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.098833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.099223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.099564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.100479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.103660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.230029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 795/1571] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.230867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.239932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:04.242591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 777/1572] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/projects => generated 375 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.242591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.298689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb349ba-f2a8-4419-a5df-40c911806d77 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.324849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b31ba-18dc-4875-b6da-6e0b76e992f4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.329669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.331737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.331737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.331737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.331737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.333562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.367293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.367987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.368288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.368628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.369445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.371523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25c138-62c4-4d2f-b473-158717333d9b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.413871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 778/1573] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/projects => generated 389 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.418382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.464714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 796/1574] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/projects => generated 371 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.476786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.476786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.476786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.477127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:04.485627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:04.486872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:04.500270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862acfd7-448c-4cda-878f-b228b9b5c37f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.554040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 797/1575] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:04.554927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.561450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.562610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.563036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.563736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.564606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.567596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.568901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86770cd6-92ca-4b00-b38c-32d9e37a309a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.574240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.575369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.575369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.575726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.577015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.578490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:04.578490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:25:04.578490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:25:04.580057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.656896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 779/1576] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.657550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.668081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2174162-b27d-4d6b-bb62-617f603745ee admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.674135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.675235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.675733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.676233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.677539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.681156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.688743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 798/1577] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.688743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.699055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952cc41-d7c3-4f31-aaad-42b046127079 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.706416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.707102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.707442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.707728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.708506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.711101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.811840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 780/1578] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.812773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.836995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 799/1579] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:04.836995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:04.841317 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:04.845759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01a07f-a369-4237-996d-60a45640a613 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.878405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9b1a1-5864-451a-904f-ab50727cc116 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:04.883900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.884755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.885043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.885326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.886024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.888000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:04.903051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:04.903731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:04.904024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:04.904333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:04.905032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:04.907043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.016728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 781/1580] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:05.017567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.027876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f68fa44-c918-4e5c-9ebd-71af41b4c4f7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.032935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 800/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:04 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:05.032935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.037362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.038064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.038376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.038858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.039713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.041746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.090679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7d971-6dc0-4454-ab47-2268eb617a1f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.094277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.095087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.095374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.095656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.105472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.107222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.118197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 801/1582] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:05.119074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.130045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402a39e-3e6d-4830-993f-6b5007bc0574 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.136036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.137202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.137202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.137532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.138050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.139712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.151331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 782/1583] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:05 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:05.156399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 802/1584] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.156651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.156813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.163363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32822c69-db9c-4450-ad11-194618b21313 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.164056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfe30b-458b-4ad1-a464-f5d8cd1efcb0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.168187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.168799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.169034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.169330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.169456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.169607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.169874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.170286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.170604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.171007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.172232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.173125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.184402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 783/1585] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.188177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.198758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87d25f-5453-466a-bb64-969c81a65ab6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.205617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.205617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.205617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.205617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.205984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.207076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.212568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:05.218139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 784/1586] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.218941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.228931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc165-925c-4439-972b-de90999bcddb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.234126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.234827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.235117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.235400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.236145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.237827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.246782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 803/1587] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:05 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:05.249920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.250060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 785/1588] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.253394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.260352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb12671-4f09-44f3-afa4-e653661234cf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.265136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951eba7-6f83-41c2-a4fa-e0769ce4fd74 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.265918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.266841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.267350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.267707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.268582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.270294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.271834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.273326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.273692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.273983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.274722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.276398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.283139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 804/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.283986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.296655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 786/1590] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.296859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e56be4-071f-4f2a-a9f6-5df16f6b17b2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.296975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.300062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.300872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.301221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.301572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.302318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.304622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.310237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060603d-7c99-4a6d-a32e-05205c85612e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.319508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 805/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.319508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.324780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.324780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.324780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.324780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.325170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.325170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.332443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42356ea-0b36-4dd9-b094-649cf0cc950f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.335874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 787/1592] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.335874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.340882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.341503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.341930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.342087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.342784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.345917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.348778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95579b-51ed-4fc6-b273-685e9d4b0ffc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.354839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.355522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.358954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 806/1593] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.359100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.359100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.359279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.359371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.359371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.367448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca983ca7-dda1-4df2-9b33-cd4e67ea33e0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.369902 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 788/1594] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.370757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.374148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.375104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.375421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.375709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.376508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.379504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.380794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7b5fb-cfc0-4855-b26c-c3cd696d73b0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.386720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.387346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.387638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.387928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.388696 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.390324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.393111 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 807/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.393972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.401180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 789/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.401883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.406307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ef3cf-1d97-4dee-9da9-3711e6f6f85c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.415773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.416513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.416803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.417349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.418917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.420543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.421330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2d9ba-ef5b-40d2-b2e2-b27111f3ef27 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.428057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.430830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 808/1597] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.431059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.431416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.431807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.431976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] PATH_INFO: `/v3/projects/d245c06c21604eff89d4b21cce3d657c/users/bcc57a9a38d14a79b27989dc57be4f4a/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.456139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1adbf-bdff-4f5e-b7f6-b8d8b8d0b37e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.462404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.464621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.464621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.465115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] PATH_INFO: `/v3/projects/3ee7ece325794a1182addebaaadda780/users/e387020748424930b277ff88372bf474/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.465290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d245c06c21604eff89d4b21cce3d657c, user_id=bcc57a9a38d14a79b27989dc57be4f4a, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.467238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:05.467238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:25:05.467238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:25:05.471480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.483078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:05.488066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee7ece325794a1182addebaaadda780, user_id=e387020748424930b277ff88372bf474, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.490891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.575849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 809/1598] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/3ee7ece325794a1182addebaaadda780/users/e387020748424930b277ff88372bf474/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:05.578439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 790/1599] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/d245c06c21604eff89d4b21cce3d657c/users/bcc57a9a38d14a79b27989dc57be4f4a/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:05.581113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.586533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.676498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6be6ab-944e-4241-b925-bb4a78445b21 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.682787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.683059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9e9d-2aa0-407c-82a6-f75445520997 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.683494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.683754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.684028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.684807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.686447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.688004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.689037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.689686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.689686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] PATH_INFO: `/v3/projects/5ce90d7b4e2540d8a3417db6900d9243/users/e9f4e1de59734b3f87a982a50b5b2f91/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.705196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 791/1600] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:05.706793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.710907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ce90d7b4e2540d8a3417db6900d9243, user_id=e9f4e1de59734b3f87a982a50b5b2f91, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.715996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.726558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21574-1cb3-486b-ac29-d0885dcb1746 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.734958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.736455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.736892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.737330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] PATH_INFO: `/v3/projects/d00edd8bebc34098919b25aafca8974b/users/29bc0d7e30794a6abb292dc65329ed9e/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.765176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00edd8bebc34098919b25aafca8974b, user_id=29bc0d7e30794a6abb292dc65329ed9e, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.771076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:05.795896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 810/1601] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/5ce90d7b4e2540d8a3417db6900d9243/users/e9f4e1de59734b3f87a982a50b5b2f91/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:05.796981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.866505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 792/1602] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/d00edd8bebc34098919b25aafca8974b/users/29bc0d7e30794a6abb292dc65329ed9e/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:05.867963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:05.897536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e9c0b-e76e-4767-a259-c2c3b46a2565 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.936505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:05.949844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197eea5e-2e74-4b2d-99d9-fb49bfbcb883 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:05.954898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.955851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.956156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.956497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] PATH_INFO: `/v3/projects/c780c3e3d141486cbcd8250dae457e32/users/815b12a77d4d4649804d5327ddeaca62/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.983327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:05.983929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c780c3e3d141486cbcd8250dae457e32, user_id=815b12a77d4d4649804d5327ddeaca62, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:05.984247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:05.984567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:05.984855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] PATH_INFO: `/v3/projects/775c525b714e4f13ac5f34314243fe93/users/d5148114d71b401ca20f988485fb6c88/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:05.986874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb1e7f-3bd4-48fe-b783-ddb66b18e20d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.003801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775c525b714e4f13ac5f34314243fe93, user_id=d5148114d71b401ca20f988485fb6c88, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:06.006651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684f9de-4f16-4985-a00f-7791c9f7dc70 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.080517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 793/1603] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/c780c3e3d141486cbcd8250dae457e32/users/815b12a77d4d4649804d5327ddeaca62/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:06.082080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.082383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.082708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.122839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] MFA Rules not processed for user `e387020748424930b277ff88372bf474`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.127649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.138301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 811/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:05 2022] PUT /identity/v3/projects/775c525b714e4f13ac5f34314243fe93/users/d5148114d71b401ca20f988485fb6c88/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:06.138301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821a9e-495a-4a9e-9b30-2862697d27f4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.138301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821a9e-495a-4a9e-9b30-2862697d27f4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.138301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35821a9e-495a-4a9e-9b30-2862697d27f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.145419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:06.173068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-35821a9e-495a-4a9e-9b30-2862697d27f4 None None] MFA Rules not processed for user `bcc57a9a38d14a79b27989dc57be4f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.174349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-35821a9e-495a-4a9e-9b30-2862697d27f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.221426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 794/1605] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.222627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:06.287991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 812/1606] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.301291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccae-5344-4d70-a461-9fc2356d9716 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.301907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccae-5344-4d70-a461-9fc2356d9716 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.302365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cccae-5344-4d70-a461-9fc2356d9716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.329181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42a462-2f96-4f5e-b07b-c866f311529d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:06.331184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7a5cccae-5344-4d70-a461-9fc2356d9716 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.332684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7a5cccae-5344-4d70-a461-9fc2356d9716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.336799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:06.338056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.338469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.338813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] PATH_INFO: `/v3/projects/dd547ab6ff5c4dcf9e68cd318e5149e0/users/d15bf5283ba443f18d3b80a051bd177c/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.347588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 813/1607] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.347588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ec88d-9eb6-45d9-8f47-f6f269e637a1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.347588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ec88d-9eb6-45d9-8f47-f6f269e637a1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.347588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ec88d-9eb6-45d9-8f47-f6f269e637a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.383126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd547ab6ff5c4dcf9e68cd318e5149e0, user_id=d15bf5283ba443f18d3b80a051bd177c, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:06.383764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-276ec88d-9eb6-45d9-8f47-f6f269e637a1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.386086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-276ec88d-9eb6-45d9-8f47-f6f269e637a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.398928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10883bab-c772-4324-a4b1-d1675992811d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.409315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 814/1608] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.409315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b347c-6f99-4c25-8bc7-64a2a5edf376 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.409315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b347c-6f99-4c25-8bc7-64a2a5edf376 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.409315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b347c-6f99-4c25-8bc7-64a2a5edf376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.436632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-310b347c-6f99-4c25-8bc7-64a2a5edf376 None None] MFA Rules not processed for user `815b12a77d4d4649804d5327ddeaca62`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.437855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-310b347c-6f99-4c25-8bc7-64a2a5edf376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.457088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 795/1609] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:06 2022] PUT /identity/v3/projects/dd547ab6ff5c4dcf9e68cd318e5149e0/users/d15bf5283ba443f18d3b80a051bd177c/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:06.458692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.458985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.459264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.480140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.481338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.526173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 815/1610] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.530106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:06.531238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85045-946f-4deb-976b-f9ff3ea0bb81 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.531402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85045-946f-4deb-976b-f9ff3ea0bb81 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.531615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf85045-946f-4deb-976b-f9ff3ea0bb81 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.533006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 816/1611] 10.222.0.47 () {54 vars in 961 bytes} [Tue Jul 12 18:25:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:06.539804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14119b-b86d-4229-8bee-ebc8cf3deaab None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.540127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14119b-b86d-4229-8bee-ebc8cf3deaab None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.540440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14119b-b86d-4229-8bee-ebc8cf3deaab None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.541777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 817/1612] 10.222.0.47 () {54 vars in 961 bytes} [Tue Jul 12 18:25:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:06.542498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14119b-b86d-4229-8bee-ebc8cf3deaab None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:06.549334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 796/1613] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.549334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:06.610059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14119b-b86d-4229-8bee-ebc8cf3deaab None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:06.615048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:06.615728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.620428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.620428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.622395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281d3b1-b9c1-4b98-b3a1-b30847d5b721 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:06.629259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:06.629544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:06.630643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.630643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.630643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.631507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169487-4250-4cca-b111-4942f93dc9fa service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.642359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:06.643836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.646753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 818/1614] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:06.649843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f6dd-b9c9-470d-ada0-3b0bea5b8cf0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.655578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f6dd-b9c9-470d-ada0-3b0bea5b8cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.655578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f6dd-b9c9-470d-ada0-3b0bea5b8cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.658162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 797/1615] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:06.659014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:06.670158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480cf23-dffb-49b2-9792-63e54594e518 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:06.672808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:06.675968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:06.676990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.677387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.677775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.678586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:06.681647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8332a-b860-4445-82df-1db00e532376 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:06.683130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a6c7f6dd-b9c9-470d-ada0-3b0bea5b8cf0 None None] MFA Rules not processed for user `d15bf5283ba443f18d3b80a051bd177c`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.684762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a6c7f6dd-b9c9-470d-ada0-3b0bea5b8cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.774357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 819/1616] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.776837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.777327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.777890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.811258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 798/1617] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:06.813210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:06.813764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:06.814274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:06.849066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.850471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.860554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:06.862452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:06.975294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 820/1618] 10.222.0.47 () {58 vars in 1058 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:06.977686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.053208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 799/1619] 10.222.0.47 () {58 vars in 1058 bytes} [Tue Jul 12 18:25:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:07.058217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.113672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050b66-438c-44a8-9f4a-8c5dccd37d99 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.121406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.123093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.123722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.124550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.189827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:07.197796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d5f44-b7ba-45ea-9a46-24511cd09469 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.203938 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.204672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.205013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.205566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.206455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.208742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.223437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:07.277887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.279805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.294746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 821/1620] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:07.295713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.326444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4c3f2-22d2-4961-ab28-f0c9dad67c78 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.336677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 800/1621] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:07 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:07.337677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.344557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.346163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.347113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.347113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.349970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.358386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.482575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 822/1622] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:07 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:07.483737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.517030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d202e-4389-46dd-a35f-0d29b0f0bb37 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.517406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a72fa-8905-4844-a6d1-8b6da53c2349 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.531609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.532568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.533383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.535056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.535056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.537863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.539390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.539714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.699243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.700764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.708179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.710657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.733983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 801/1623] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:07.735409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.739270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 823/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:07.744537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:07.921401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695b196-15fc-489f-92e9-a5e67838ef68 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.937165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.937891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.938187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.938473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.939490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.941761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.954125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a314a8-4457-4f23-ba63-86b926964769 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:07.960010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:07.961558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:07.961855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:07.962136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:07.964100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:07.967259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:07.984494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 824/1625] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:07.985484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.016554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb020-6b2d-498b-8e28-bb3cf3d8f888 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.023456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.024462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.024754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.025156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] PATH_INFO: `/v3/projects/38fc74ec2547460aaf7056068bd30668/users/26417e93b1cf4d48ad0d67b140230a06/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.045956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fc74ec2547460aaf7056068bd30668, user_id=26417e93b1cf4d48ad0d67b140230a06, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.049784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.083843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 802/1626] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:07 2022] POST /identity/v3/users => generated 297 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:08.088657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.108905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc26f6-e119-4673-90ec-0eee786736e4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.122374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.123178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.123470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.123756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.124924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.132914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.134986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:08.159786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 803/1627] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:08.163916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.183465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49d372-f25d-471a-8f92-103745acec77 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.189632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 825/1628] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:07 2022] PUT /identity/v3/projects/38fc74ec2547460aaf7056068bd30668/users/26417e93b1cf4d48ad0d67b140230a06/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:08.194067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.281570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:08.352346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.353508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.353798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.354085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] PATH_INFO: `/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee/users/8de2c87937814f659c0faabd9c2008db/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.370994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648ba54-b1b9-4938-b045-6ccb3dc93a94 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.381162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5926c95e1d4ef983b61ab0b66302ee, user_id=8de2c87937814f659c0faabd9c2008db, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.384975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fdae4-f68a-4d60-bbdc-e6bd5e1a0a29 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.395599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.396763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.398523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.400413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.402683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.408501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064d479-2571-4e0b-b551-7cd2f2c8fcb5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.442003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 826/1629] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:08.442003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8a1a8-ccb6-4537-81f6-ac47c7b0812f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.445292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8a1a8-ccb6-4537-81f6-ac47c7b0812f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.445885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8a1a8-ccb6-4537-81f6-ac47c7b0812f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.450517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 827/1630] 10.222.0.47 () {54 vars in 912 bytes} [Tue Jul 12 18:25:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:08.454100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12bfb-fa54-468e-b100-f7f8947b5930 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.454100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12bfb-fa54-468e-b100-f7f8947b5930 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.458944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a12bfb-fa54-468e-b100-f7f8947b5930 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.465118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 828/1631] 10.222.0.47 () {54 vars in 912 bytes} [Tue Jul 12 18:25:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:08.467239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12bfb-fa54-468e-b100-f7f8947b5930 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.468371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 804/1632] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:08 2022] PUT /identity/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee/users/8de2c87937814f659c0faabd9c2008db/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:08.472905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.472905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.472905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.533594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:08.538070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:08.649611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a12bfb-fa54-468e-b100-f7f8947b5930 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.655176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.656204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.656583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.656930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] PATH_INFO: `/v3/projects/38fc74ec2547460aaf7056068bd30668/users/26417e93b1cf4d48ad0d67b140230a06/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.659025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fc74ec2547460aaf7056068bd30668, user_id=26417e93b1cf4d48ad0d67b140230a06, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.662592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e44383-df5f-4c7c-bf3a-21785130ab94 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.684057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 829/1633] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:08 2022] PUT /identity/v3/projects/38fc74ec2547460aaf7056068bd30668/users/26417e93b1cf4d48ad0d67b140230a06/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:08.685717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb4bc-bac2-4380-9495-d42e35a72a3d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.686013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb4bc-bac2-4380-9495-d42e35a72a3d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.686447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbb4bc-bac2-4380-9495-d42e35a72a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.709931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 805/1634] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:08.713203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.728666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180aab3f-4f5d-4a29-aa6c-ed79b4786c00 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.731464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-37fbb4bc-bac2-4380-9495-d42e35a72a3d None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:08.732822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-37fbb4bc-bac2-4380-9495-d42e35a72a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:08.827867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.828520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.828871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.829168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.829989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.832130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:08.834398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 830/1635] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:08.839476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.841215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.842634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.857783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 806/1636] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:08.860989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:08.891795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] MFA Rules not processed for user `26417e93b1cf4d48ad0d67b140230a06`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:08.893793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:08.947165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa705e-2d12-42f7-a6d8-8c3cdb735af7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:08.951218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:08.951391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:08.951884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:08.952196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:08.958240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:08.979644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:08.984190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:09.003886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 807/1637] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:09.004784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:09.019428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aced43a-89fc-4851-8252-e3fa0a08ea8c service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:09.028752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:09.029615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:09.030005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:09.030418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:09.059589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 831/1638] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:09.068573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:09.070284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:09.072026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05461bb-0b0c-412b-8421-af0382e24134 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:09.100045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6385a-c5f5-4bcd-be09-a32e083ad2c8 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:09.110472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 808/1639] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:09.136498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:09.143945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:09.145245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:09.148583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] PATH_INFO: `/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee/users/8de2c87937814f659c0faabd9c2008db/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:09.154032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e5926c95e1d4ef983b61ab0b66302ee, user_id=8de2c87937814f659c0faabd9c2008db, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:09.159420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:09.191776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 832/1640] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:09 2022] PUT /identity/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee/users/8de2c87937814f659c0faabd9c2008db/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:09.203521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:09.204132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:09.204621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:09.286565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] MFA Rules not processed for user `8de2c87937814f659c0faabd9c2008db`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:09.290569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:09.361948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:09.447519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 809/1641] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:10.554648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:10.700772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862a3a-7366-4f3d-950b-5540e0bf9834 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:11.020224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:11.020224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:11.020224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:11.020224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:11.046349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:11.046349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4675c4a-a743-44f4-a463-19ebce50240d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:11.046349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:11.046349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:11.049699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:11.049699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:11.049699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:11.049699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:11.049699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 810/1642] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:11.053047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:11.053047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:11.053047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 833/1643] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:11.594340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:11.601917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a528-c453-4c52-8880-b006815ef786 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:11.614560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:11.614723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:11.615077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:11.617209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:11.620115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:11.629993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624777e-f903-4bfc-a6af-e39dee9626b5 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:11.645797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:11.648843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:11.649069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:11.649363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:11.688663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:11.690234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dae12-2324-4a41-ad02-05eaf8d52c2c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:11.702543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 811/1644] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:11.836878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:11.839401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42451639-3547-46ce-baa4-e4b2ba9e6099 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:11.862557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 834/1645] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:15.198029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:15.198579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:15.198579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:15.236050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:15.273534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] MFA Rules not processed for user `b58b0d520aa1418bb6ad3b5645a826a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:15.277645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:15.523691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 812/1646] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:15.533726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:15.534733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:15.534733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:15.545445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:15.577413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] MFA Rules not processed for user `b58b0d520aa1418bb6ad3b5645a826a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:15.579815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:15.611697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 835/1647] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:15.763977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:15.798866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:16.040136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbb9c8-34a3-4c1b-bb08-05d0a08c7d02 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:16.054437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:16.061861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:16.063664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:16.064489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:16.095981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:16.098314 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcad47-72ee-487e-943c-c31bfc32b9d0 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:16.100967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65cd90-149d-4807-94b4-1ea2af5e682b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:16.124901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:16.131398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:16.132546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:16.134182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] PATH_INFO: `/v3/users/bcc57a9a38d14a79b27989dc57be4f4a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:16.134633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 813/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:16.149334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc57a9a38d14a79b27989dc57be4f4a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:16.151826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:16.470945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc57a9a38d14a79b27989dc57be4f4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:16.492388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc57a9a38d14a79b27989dc57be4f4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:16.528204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc57a9a38d14a79b27989dc57be4f4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:16.528204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc57a9a38d14a79b27989dc57be4f4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:16.529060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc57a9a38d14a79b27989dc57be4f4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:16.632525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 836/1649] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:15 2022] DELETE /identity/v3/users/bcc57a9a38d14a79b27989dc57be4f4a => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:16.641161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2c08e-0116-42b6-8e0d-4f722428600b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:16.641693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2c08e-0116-42b6-8e0d-4f722428600b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:16.641782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2c08e-0116-42b6-8e0d-4f722428600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:16.695655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-beb2c08e-0116-42b6-8e0d-4f722428600b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:16.699971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-beb2c08e-0116-42b6-8e0d-4f722428600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:16.882802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 814/1650] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:16.911657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:16.914323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:17.055840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a751104-8336-49f2-929d-aa7c850fade7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:17.065066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:17.070568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:17.070568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:17.070568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:17.090481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:17.091946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205a900-7a1f-4b3e-a37b-b808a4ea25f6 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:17.126269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 837/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:17.680494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec252c2-8366-4562-8133-5329ea712f7c None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:17.682383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec252c2-8366-4562-8133-5329ea712f7c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:17.683203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec252c2-8366-4562-8133-5329ea712f7c None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:17.686836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 815/1652] 10.222.0.47 () {54 vars in 912 bytes} [Tue Jul 12 18:25:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:17.695430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:17.700924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:17.700924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:17.773501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:17.780836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:17.783151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:17.801147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 838/1653] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:25:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:17.831723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec252c2-8366-4562-8133-5329ea712f7c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:17.886949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec252c2-8366-4562-8133-5329ea712f7c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:17.895205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:17.896524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:17.897066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:17.899789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:17.911569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:17.914987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f295e-3d8e-41eb-937b-9a591959618a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:17.929101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 816/1654] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:19.263779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:19.266947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:19.339887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02d2c4-591c-4d1e-a5cd-86dab90c3fce None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:19.369043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:19.375443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:19.375994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:19.378138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] PATH_INFO: `/v3/projects/d245c06c21604eff89d4b21cce3d657c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:19.380125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d245c06c21604eff89d4b21cce3d657c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:19.387308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: WARNING py.warnings [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:19.387308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: warnings.warn(msg) Jul 12 18:25:19.387308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: Jul 12 18:25:19.388577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:19.630909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d959-f5d2-466b-ad3a-a6344f467ba5 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:19.631387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d959-f5d2-466b-ad3a-a6344f467ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:19.631620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d959-f5d2-466b-ad3a-a6344f467ba5 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:19.634065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 817/1655] 10.222.0.47 () {54 vars in 960 bytes} [Tue Jul 12 18:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:19.643576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03566-6528-46f6-888b-6ed76bf108e7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:19.643881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03566-6528-46f6-888b-6ed76bf108e7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:19.644194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a03566-6528-46f6-888b-6ed76bf108e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:19.657344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd245c06c21604eff89d4b21cce3d657c', 'request_id': 'req-144510e4-1b61-4326-a396-6b6c8574cafc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:19.721832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd245c06c21604eff89d4b21cce3d657c', 'request_id': 'req-144510e4-1b61-4326-a396-6b6c8574cafc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:19.744946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c2a03566-6528-46f6-888b-6ed76bf108e7 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:19.746218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c2a03566-6528-46f6-888b-6ed76bf108e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:19.761884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:19.827212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd245c06c21604eff89d4b21cce3d657c', 'request_id': 'req-144510e4-1b61-4326-a396-6b6c8574cafc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:19.851712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] The token cache is being invalidate because project d245c06c21604eff89d4b21cce3d657c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:19.851928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:19.853712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-144510e4-1b61-4326-a396-6b6c8574cafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:19.869768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 839/1656] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:19 2022] DELETE /identity/v3/projects/d245c06c21604eff89d4b21cce3d657c => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:19.875683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:19.878900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:19.880184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:19.953429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] MFA Rules not processed for user `b58b0d520aa1418bb6ad3b5645a826a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:19.955777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:19.959266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 818/1657] 10.222.0.47 () {58 vars in 1057 bytes} [Tue Jul 12 18:25:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:19.963098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d2b8-64d2-4d6c-acf8-ebe848900d69 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:19.963599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d2b8-64d2-4d6c-acf8-ebe848900d69 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:19.968168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d2b8-64d2-4d6c-acf8-ebe848900d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.014661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2580d2b8-64d2-4d6c-acf8-ebe848900d69 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:20.018593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2580d2b8-64d2-4d6c-acf8-ebe848900d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:20.145211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 840/1658] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:20.146228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:20.184113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9ee45-7f30-4b19-bfef-9e86b8437e50 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:20.194501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 819/1659] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:20.202732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:20.203557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:20.203999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.209053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:20.214780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:20.214780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:20.214984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.219751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:20.251564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:20.254435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:20.283491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 841/1660] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:20.288916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:20.290448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:20.300612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 820/1661] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:20.306474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:20.319866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d3710-1cca-4bfe-953d-0db15785f1b5 service placement] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:20.326828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:20.327724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:20.328188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:20.328648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.329575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:20.330063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:20.332671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:20.369699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:20.377524 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 842/1662] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:20.396208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7be8dc-a96c-4135-8e02-b95ed860edfb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:20.410442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:20.411130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:20.411570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:20.412009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.412981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:20.415718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:20.593881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 821/1663] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:20 2022] POST /identity/v3/projects => generated 387 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:20.598170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:20.618529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d7c5c-6ba0-4210-ae7f-0401c3b959a0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:20.899290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:20.901834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:20.902417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:20.903196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:20.904179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:20.906814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:20.919798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:21.059001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 843/1664] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:20 2022] POST /identity/v3/users => generated 294 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:21.059931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:21.099564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07a47c-c33c-4748-89e2-a6d994ebd445 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:21.115775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:21.119746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.124813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.124813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.125372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:21.127213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:21.159338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 822/1665] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:21.163866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:21.183316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e860276-3cb9-4099-8580-24fcf0bc20b8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:21.197601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:21.198286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.198588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.199201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.199646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:21.201433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e399d3f-4161-45fe-b3db-8899717fca8b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:21.226071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 844/1666] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:21.228891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:21.278787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82022a1-b594-4ad7-9227-3d97bbcf3bf6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:21.309057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:21.313120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.313425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.313692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] PATH_INFO: `/v3/projects/3044f3ab7eb347a38b84cea53e9e474c/users/51b978eee8ea4ea3baa9168a5bb3a40c/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.370114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3044f3ab7eb347a38b84cea53e9e474c, user_id=51b978eee8ea4ea3baa9168a5bb3a40c, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:21.372970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f4fd2-4e65-49d3-ba5d-fd503a10e497 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:21.485551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 823/1667] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:21 2022] PUT /identity/v3/projects/3044f3ab7eb347a38b84cea53e9e474c/users/51b978eee8ea4ea3baa9168a5bb3a40c/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:21.496381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.497235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.497678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.557782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] MFA Rules not processed for user `51b978eee8ea4ea3baa9168a5bb3a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:21.559313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:21.636021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9c4f-9ccc-49d6-91b6-f12cb7c6b0fa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.637869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9c4f-9ccc-49d6-91b6-f12cb7c6b0fa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.638650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd9c4f-9ccc-49d6-91b6-f12cb7c6b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.712971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 845/1668] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:21.713963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:21.754777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ddcd9c4f-9ccc-49d6-91b6-f12cb7c6b0fa None None] MFA Rules not processed for user `e9f4e1de59734b3f87a982a50b5b2f91`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:21.755826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ddcd9c4f-9ccc-49d6-91b6-f12cb7c6b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:21.860373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:21.867884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fbfc8-81b9-4339-9d97-17c1984f7cf5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:21.901665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:21.902584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.906175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.908545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:21.909499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:21.911691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d19ffd-f577-4ea8-b009-dafea849fb55 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:21.986740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 824/1669] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:21.991534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:21.991534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:21.991534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.033555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] MFA Rules not processed for user `d5148114d71b401ca20f988485fb6c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:22.034753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:22.061899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 846/1670] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:22.064108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.064646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.065175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.092699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] MFA Rules not processed for user `e9f4e1de59734b3f87a982a50b5b2f91`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:22.096205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:22.215089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 847/1671] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:22.215089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.280615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 825/1672] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:22.284108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.289688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:22.406179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172303a4-cb2e-425d-b1ab-c85d0436a75e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.426740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.427610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.429696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.430429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.431858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:22.439771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:22.534771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43b86c-7435-4f15-93b4-c29675c2fbcb None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.593253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 848/1673] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:22 2022] POST /identity/v3/users => generated 294 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:22.596996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.638927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786fdb-9489-44ac-a393-13701a2000e4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.668058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.669158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.669622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.670200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.671170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:22.673439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:22.691050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.693901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.693901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.693901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.705469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 849/1674] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:25:22.706718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.713362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:22.717375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:22.719218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:22.731257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 826/1675] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:22.732206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.753249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feeb55f-58af-4d62-a516-7e91776d6877 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.765511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.766203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.766496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.766778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.767521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:22.769197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:22.777863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c3cf8-8e8c-4122-b93c-6d7c78c8f15f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.797884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 827/1676] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:22.800720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:22.813721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.813721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.813721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.813721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.821595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1fbae-3d2e-4a03-ac5d-298a54c9749f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:22.843076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:22.845261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:22.845625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:22.845986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] PATH_INFO: `/v3/projects/a11324889c744409a2f70afe872e352f/users/32418947ac8442fcb4714f28afca8bd1/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:22.868430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:22.901467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11324889c744409a2f70afe872e352f, user_id=32418947ac8442fcb4714f28afca8bd1, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:22.906083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427c6aa-44c3-44b9-b037-df4b54ef415c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:22.964283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:23.026548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 828/1677] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:22 2022] PUT /identity/v3/projects/a11324889c744409a2f70afe872e352f/users/32418947ac8442fcb4714f28afca8bd1/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:23.038057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb03ffa-0453-4081-a137-031296348dd9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.038593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb03ffa-0453-4081-a137-031296348dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.039167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb03ffa-0453-4081-a137-031296348dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:23.093533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7eb03ffa-0453-4081-a137-031296348dd9 None None] MFA Rules not processed for user `32418947ac8442fcb4714f28afca8bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:23.095680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7eb03ffa-0453-4081-a137-031296348dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:23.221085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:23.223439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:23.251421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 850/1678] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:23.336862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 829/1679] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:23.341273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:23.367926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb665a99-0e2b-4f59-8e18-b9c5e411e5b0 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:23.523465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.523735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.524089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:23.552289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:23.553078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.553474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.553827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:23.554647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:23.571277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eba9c-2339-4861-9697-aad681747057 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:23.595873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] MFA Rules not processed for user `29bc0d7e30794a6abb292dc65329ed9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:23.597551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:23.718782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 830/1680] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:23.766423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 851/1681] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:23 2022] POST /identity/v3/projects => generated 387 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:23.768936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:23.784352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe073e7b-5e5a-4660-8ac8-1fd13c62b937 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:23.849682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:23.900136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:23.900967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.901267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.901552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:23.902600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:23.904945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:23.991698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8471-4ac5-447c-a3ba-1c981f636ebd None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.991698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8471-4ac5-447c-a3ba-1c981f636ebd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.991698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa8471-4ac5-447c-a3ba-1c981f636ebd None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:23.991698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 852/1682] 10.222.0.47 () {54 vars in 912 bytes} [Tue Jul 12 18:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:23.993983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:23.994469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:23.994952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.010879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:24.057658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 831/1683] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:23 2022] POST /identity/v3/users => generated 300 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:24.061040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.085792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:24.098052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:24.121476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00715253-9178-4ed6-90c9-ff266c9282ca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.127825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.128743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.129513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.130146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.131057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.133879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.157429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 832/1684] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:24.157429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.266039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281cf61-f0b3-4ed6-b6c2-fc8aeba127f1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.281168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.281865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.282159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.282493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.325083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 853/1685] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:25:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:24.332301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.357404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad381e01-a688-48af-9e40-e429790bcc93 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.373026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.374387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.374856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.375479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] PATH_INFO: `/v3/projects/dd5cddf6345149f0ab42a206bf9bc848/users/0bc33adeb5f547b0bc8ad804714f0fff/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.413543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5cddf6345149f0ab42a206bf9bc848, user_id=0bc33adeb5f547b0bc8ad804714f0fff, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.416901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.477401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.481595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.501044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 833/1686] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:24.502450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.504243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:24.528795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 854/1687] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:24 2022] PUT /identity/v3/projects/dd5cddf6345149f0ab42a206bf9bc848/users/0bc33adeb5f547b0bc8ad804714f0fff/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:24.530278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.547486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce3bb2-2b7c-4dad-af90-c3d07b807bae service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.681853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:24.698774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.699642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.700611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.701840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.716017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed292547-313a-47ab-9725-6b79c638ac02 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.724942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.725785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.727370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.727370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.727370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.730511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.732735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.734277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea74d13-a7b3-4a5c-928b-4efcbac98b2a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.747649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 855/1688] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:24.748666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:24.763198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d69a-7e98-4430-b4b5-4552f7cb5b00 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:24.767022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 834/1689] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:24.771666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:24.786540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.787143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.788775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] PATH_INFO: `/v3/projects/dd5cddf6345149f0ab42a206bf9bc848/users/0bc33adeb5f547b0bc8ad804714f0fff/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.793364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5cddf6345149f0ab42a206bf9bc848, user_id=0bc33adeb5f547b0bc8ad804714f0fff, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:24.800757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:24.823824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 856/1690] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:24 2022] PUT /identity/v3/projects/dd5cddf6345149f0ab42a206bf9bc848/users/0bc33adeb5f547b0bc8ad804714f0fff/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:24.826212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:24.826517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:24.826947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:24.861162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] MFA Rules not processed for user `0bc33adeb5f547b0bc8ad804714f0fff`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:24.862707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:24.970343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 835/1691] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:24 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:24.998827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:25.000731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:25.109386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15deaa-1d50-4cb8-9f27-6568e2c7a680 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:25.117846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:25.118799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:25.119237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:25.119709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:25.134081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:25.136291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb285d0-90a7-494b-a869-1063f7064dfd service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:25.150846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 857/1692] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:27.304060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:27.329041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd38ff-365a-4e5f-ac3b-c78c6946e021 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:27.340115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:27.340943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:27.341227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:27.341681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:27.501075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:27.502556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55598eff-abac-47f1-8668-a5626440b201 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:27.524703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 836/1693] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:28.862210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:28.863127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:28.863656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:28.905628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] MFA Rules not processed for user `d5148114d71b401ca20f988485fb6c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:28.907804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:28.923254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:28.989925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:28.990449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:28.990789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:29.190873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:29.192624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:29.196747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 858/1694] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:29.351122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:29.471751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 837/1695] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:25:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:29.551724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:29.554509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:29.735516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c6a77-8d35-482c-b85b-889623036785 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:29.743947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:29.745356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:29.745535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:29.745921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] PATH_INFO: `/v3/users/e387020748424930b277ff88372bf474` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:29.748012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e387020748424930b277ff88372bf474)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:29.750421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:30.135860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e387020748424930b277ff88372bf474'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:30.163850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e387020748424930b277ff88372bf474'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:30.190546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:30.192337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:30.211324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e387020748424930b277ff88372bf474'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:30.216437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e387020748424930b277ff88372bf474'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:30.218766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-55bf51ce-b0ba-4bc2-a154-544cad27733e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e387020748424930b277ff88372bf474'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:30.330983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 859/1696] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:29 2022] DELETE /identity/v3/users/e387020748424930b277ff88372bf474 => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:30.336867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:30.337467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:30.340487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:30.406686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:30.416058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:30.559012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b27a4-2894-439a-ad5c-2632774ee70b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:30.685078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 860/1697] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:30.789368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:30.789451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:30.789451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:30.789451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:30.789451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:30.789728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:30.811022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:30.959866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b057ff-69f7-4b03-8557-60e956a3462f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:30.967122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:30.967792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:30.968084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:30.968399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:30.999943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:31.001324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153e27c-da14-4470-9ebe-64919e957d9c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:31.018701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 861/1698] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:31.027360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:31.029843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:31.055623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 838/1699] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:25:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 865 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:25:31.317977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209fb3-17ca-407b-8123-95310a63627e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:31.318552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209fb3-17ca-407b-8123-95310a63627e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:31.318552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f209fb3-17ca-407b-8123-95310a63627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:31.425469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f209fb3-17ca-407b-8123-95310a63627e None None] MFA Rules not processed for user `29bc0d7e30794a6abb292dc65329ed9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:31.426729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f209fb3-17ca-407b-8123-95310a63627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:31.449976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:31.723042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 862/1700] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:32.507399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:32.509967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:32.568823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168d478-b981-4327-96e9-1894f74bddba service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:32.597916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:32.605739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:32.606164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:32.606457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] PATH_INFO: `/v3/users/d15bf5283ba443f18d3b80a051bd177c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:32.610217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15bf5283ba443f18d3b80a051bd177c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:32.611300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: WARNING py.warnings [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 18:25:32.611300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: warnings.warn(msg) Jul 12 18:25:32.611300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: Jul 12 18:25:32.620399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:32.640961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:33.316490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:33.317446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:33.317880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:33.373885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:33.376141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:33.378307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15bf5283ba443f18d3b80a051bd177c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:33.379806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15bf5283ba443f18d3b80a051bd177c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:33.412710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 863/1701] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Jul 12 18:25:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:33.451291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15bf5283ba443f18d3b80a051bd177c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:33.472609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:33.474051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:33.475262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15bf5283ba443f18d3b80a051bd177c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:33.504388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ea151-b5ef-4e90-b9c1-e2da5d3b632c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:33.509685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:33.531211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:33.531862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:33.532156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:33.545264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:33.546729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69321b-6bce-4f8f-982f-4b3d0bde777d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:33.593421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 864/1702] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:33.637783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15bf5283ba443f18d3b80a051bd177c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:33.645991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 839/1703] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:32 2022] DELETE /identity/v3/users/d15bf5283ba443f18d3b80a051bd177c => generated 0 bytes in 1140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:33.653960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:33.653960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:33.653960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:33.687854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:33.694657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:33.950023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 865/1704] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:34.056852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:34.058444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:34.072073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:34.108509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc4abd-6076-421d-8a92-8db07718d36f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:34.120285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:34.123390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.123390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.123390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] PATH_INFO: `/v3/projects/3ee7ece325794a1182addebaaadda780` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.125330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee7ece325794a1182addebaaadda780)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:34.129417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:34.177841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebc291-68ff-4fa7-a449-9e25c1a19f26 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:34.182827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:34.183895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.184381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.185013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.204749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:34.204749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4ba074-77db-47bb-a486-f6d88e27d641 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:34.210630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 866/1705] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:34.376662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee7ece325794a1182addebaaadda780', 'request_id': 'req-08c8374d-0fe5-49e0-842c-3d4fd6931223'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:34.394493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee7ece325794a1182addebaaadda780', 'request_id': 'req-08c8374d-0fe5-49e0-842c-3d4fd6931223'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:34.517614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee7ece325794a1182addebaaadda780', 'request_id': 'req-08c8374d-0fe5-49e0-842c-3d4fd6931223'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:34.551112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] The token cache is being invalidate because project 3ee7ece325794a1182addebaaadda780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:34.551694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:34.552603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08c8374d-0fe5-49e0-842c-3d4fd6931223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:34.558327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 840/1706] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:34 2022] DELETE /identity/v3/projects/3ee7ece325794a1182addebaaadda780 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:34.566317 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.566317 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.566317 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.607230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:34.608976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:34.785201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 867/1707] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:34.788064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.788845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.789089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.829923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:34.832214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:34.858820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 841/1708] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:34.863022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:34.873924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5726b03f-792c-4b04-95d1-b2f7099fde95 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:34.880804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:34.881872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.882262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.882554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.884785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:34.887410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:34.887410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:34.908731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 868/1709] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:34.909449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:34.924863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df361a-ae9a-4ed7-9cbc-37961ad9a98e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:34.934932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:34.936075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:34.936566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:34.937018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:34.938380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:34.942738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:35.101576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 842/1710] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:34 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:35.103219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:35.120466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad54f-1955-4382-8083-90fe50bd15b5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:35.374571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:35.380056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:35.380056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:35.380056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:35.380056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:35.380056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:35.570050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 869/1711] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:35 2022] POST /identity/v3/users => generated 283 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:35.574975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:35.589645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0dc92-107b-4ad7-8a4e-4f1caba6deb3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:35.595995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:35.597174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:35.597638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:35.598320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:35.599614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:35.605874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:35.623576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 843/1712] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:35.638168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:35.664788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0292f-3821-493e-8da2-db3bb5680960 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:35.670235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:35.670992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:35.671348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:35.671693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:35.672680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:35.674770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:35.693780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 870/1713] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:35.699023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:35.711522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe489c-daf1-48c2-b8ce-77a19af41c9f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:35.717095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:35.718141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:35.718409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:35.718734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] PATH_INFO: `/v3/projects/dc9bd5454a8948a28ce05392ff2e13e0/users/641ade3664db41dca1613ce8eac5d0e8/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:35.743039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9bd5454a8948a28ce05392ff2e13e0, user_id=641ade3664db41dca1613ce8eac5d0e8, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:35.752402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c9727-a174-469d-af8b-cde07e304dd9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:35.779247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:35.779247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:35.989071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00b952-8a1b-470e-872e-f8cdefc49cce admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:36.000400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:36.001961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:36.008283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:36.008283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:36.008283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:36.042332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=79,keystone:DELETE=17,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:36.057990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 844/1714] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:35 2022] PUT /identity/v3/projects/dc9bd5454a8948a28ce05392ff2e13e0/users/641ade3664db41dca1613ce8eac5d0e8/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:36.065055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0fc2-f607-4f33-b7e9-fd6f4cbf412c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:36.066043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0fc2-f607-4f33-b7e9-fd6f4cbf412c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:36.066043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44f0fc2-f607-4f33-b7e9-fd6f4cbf412c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:36.140296 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:36.204863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a44f0fc2-f607-4f33-b7e9-fd6f4cbf412c None None] MFA Rules not processed for user `641ade3664db41dca1613ce8eac5d0e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:36.209841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a44f0fc2-f607-4f33-b7e9-fd6f4cbf412c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:36.277424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:36.279978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:36.330930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 871/1715] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:36.463070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 845/1716] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:36.466690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:36.484843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2ad8d-5a16-4f4d-a14b-76d50f1bc597 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:36.642347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf44b7-f54c-4ce4-903b-c4258c8ccfbd None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:36.643030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf44b7-f54c-4ce4-903b-c4258c8ccfbd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:36.643476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf44b7-f54c-4ce4-903b-c4258c8ccfbd None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:36.646379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 846/1717] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:36.656805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925611f6-8632-4427-be40-3a529ca16897 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:36.657372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925611f6-8632-4427-be40-3a529ca16897 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:36.658050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925611f6-8632-4427-be40-3a529ca16897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:36.671933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:36.678707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:36.678707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:36.678707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:36.678707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:36.678707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:36.698028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-925611f6-8632-4427-be40-3a529ca16897 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:36.701670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-925611f6-8632-4427-be40-3a529ca16897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:36.833817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 872/1718] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:36 2022] POST /identity/v3/projects => generated 365 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:36.839234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:36.946367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 847/1719] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:25:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:37.021251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccced95-4220-4a29-b334-60d7e64cbed8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.068120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:37.072771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.074267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.078866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.083399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:37.087765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:37.240603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 873/1720] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:36 2022] POST /identity/v3/users => generated 289 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:25:37.245411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925611f6-8632-4427-be40-3a529ca16897 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:37.250034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:37.268260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925611f6-8632-4427-be40-3a529ca16897 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.287348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:37.288731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.295962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.301533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.301533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:37.301533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:37.371708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:37.395403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 848/1721] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:37.395504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985774f7-8d9e-4d0b-83e9-419932bde57e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.395587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:37.395587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:37.402436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:37.407108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.410150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.410754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] PATH_INFO: `/v3/projects/301ab67ea9a44d808ec358604b529cd5/users/0bd8c4b14d6e4105861d3dd47e000034/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.471389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301ab67ea9a44d808ec358604b529cd5, user_id=0bd8c4b14d6e4105861d3dd47e000034, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:37.474434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:37.550878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 874/1722] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:37 2022] PUT /identity/v3/projects/301ab67ea9a44d808ec358604b529cd5/users/0bd8c4b14d6e4105861d3dd47e000034/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:37.551834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:37.731477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0d476-a11e-4b0b-b75b-64d3df7dd7c5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.736697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4193f-6aa0-46c6-969b-acdd70480d4d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.742869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:37.743619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.743915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.744276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.745228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:37.748051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:37.803927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 875/1723] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:37.805825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:37.821110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c6b3-2112-46b4-8a33-a6dfb3b21bb5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:37.829801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:37.830837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.831131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.832107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] PATH_INFO: `/v3/projects/301ab67ea9a44d808ec358604b529cd5/users/0bd8c4b14d6e4105861d3dd47e000034/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.834903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301ab67ea9a44d808ec358604b529cd5, user_id=0bd8c4b14d6e4105861d3dd47e000034, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:37.839500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988396f-5090-4cf8-b21a-ae046ef79767 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:37.870984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 876/1724] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:37 2022] PUT /identity/v3/projects/301ab67ea9a44d808ec358604b529cd5/users/0bd8c4b14d6e4105861d3dd47e000034/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:37.873546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:37.873849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:37.874142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:37.945818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] MFA Rules not processed for user `0bd8c4b14d6e4105861d3dd47e000034`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:37.947432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:38.012781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:38.024082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:38.024082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:38.024082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:38.064073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:38.065754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c8c24-4266-4975-a44c-1d765b6e67b6 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:38.107150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 849/1725] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:38.168790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:38.169010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:38.169296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:38.171180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 850/1726] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:38.186250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 877/1727] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:38.206863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:38.209717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:38.309320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:38.310979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:38.564548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fffda5-1eeb-46d1-9c1d-f10701bfa470 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:38.577482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:38.578169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:38.578451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:38.578886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:38.629446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:38.632021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb888-ce4a-47cd-af79-26d292e7a1f7 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:38.639887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36996c-6c09-48d1-abda-b6d85f3981ca None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:38.661572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:38.662984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:38.663427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:38.663875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] PATH_INFO: `/v3/projects/dd547ab6ff5c4dcf9e68cd318e5149e0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:38.665645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd547ab6ff5c4dcf9e68cd318e5149e0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:38.668075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:38.675750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 851/1728] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:38.677334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:38.677632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:38.678128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:38.744965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:38.746322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:38.867560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd547ab6ff5c4dcf9e68cd318e5149e0', 'request_id': 'req-728ff0cd-9da6-435f-9441-f519f78bdb9c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:38.897472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd547ab6ff5c4dcf9e68cd318e5149e0', 'request_id': 'req-728ff0cd-9da6-435f-9441-f519f78bdb9c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:38.941594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 852/1729] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:25:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:38.987867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd547ab6ff5c4dcf9e68cd318e5149e0', 'request_id': 'req-728ff0cd-9da6-435f-9441-f519f78bdb9c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.024920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] The token cache is being invalidate because project dd547ab6ff5c4dcf9e68cd318e5149e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:39.025133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.026503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.033360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:39.045104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 878/1730] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:38 2022] DELETE /identity/v3/projects/dd547ab6ff5c4dcf9e68cd318e5149e0 => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:39.049019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:39.061277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:39.064899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7511e3e-d154-4ceb-b49a-80bf9c60e042 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:39.316487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ff0cd-9da6-435f-9441-f519f78bdb9c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:39.329986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:39.333514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:39.334148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:39.337323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] PATH_INFO: `/v3/users/8de2c87937814f659c0faabd9c2008db` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:39.340825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:39.341500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:39.341791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:39.341926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de2c87937814f659c0faabd9c2008db)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:39.342355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:39.344542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:39.345763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:39.348290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:39.385265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:39.393948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:39.403778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:39.424908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 853/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:39.426046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:39.429714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:39.707893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de2c87937814f659c0faabd9c2008db'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.746968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de2c87937814f659c0faabd9c2008db'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.783698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053b155-3c6b-4df1-9753-bc47d9385d33 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:39.803243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:39.805858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:39.805858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:39.805858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:39.810927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:39.833243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de2c87937814f659c0faabd9c2008db'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.839976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de2c87937814f659c0faabd9c2008db'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.841308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de2c87937814f659c0faabd9c2008db'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:39.960578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 879/1732] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:39 2022] DELETE /identity/v3/users/8de2c87937814f659c0faabd9c2008db => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:40.041044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:40.052376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be4bb0-0f9e-40b0-814a-c5e8b9ff574d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:40.121091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 854/1733] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:40.193161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:41.826936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:42.360975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:42.362582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:42.590550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ae981-2276-466b-adf4-c8b6374a9158 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:42.611760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:42.614354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:42.620465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:42.620917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] PATH_INFO: `/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:42.625564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e5926c95e1d4ef983b61ab0b66302ee)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:42.628045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:42.815017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e5926c95e1d4ef983b61ab0b66302ee', 'request_id': 'req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:42.832329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e5926c95e1d4ef983b61ab0b66302ee', 'request_id': 'req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:42.957708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e5926c95e1d4ef983b61ab0b66302ee', 'request_id': 'req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:42.979083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] The token cache is being invalidate because project 4e5926c95e1d4ef983b61ab0b66302ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:42.979640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:42.983877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e7c1c2f4-21a9-476b-a11f-e3608c3c1e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:42.988898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 880/1734] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:42 2022] DELETE /identity/v3/projects/4e5926c95e1d4ef983b61ab0b66302ee => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:42.994912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:42.995376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:42.995376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.041566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:43.050288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:43.200515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 855/1735] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:43.210924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:43.213928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:43.213928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.299705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:43.304099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:43.339184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 881/1736] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:43.343571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:43.360371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bb6ef-8390-467c-809c-7c36c42f2876 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:43.378852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:43.378852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:43.378852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:43.378852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.379343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:43.379343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:43.379343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:43.380469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:43.399722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 856/1737] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:43.404243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:43.423263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f9782-8360-4e7c-974f-feeaa6fc82d9 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:43.436351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:43.437836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:43.438423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:43.438880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.439774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:43.441985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:43.591960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 882/1738] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:43 2022] POST /identity/v3/projects => generated 367 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:43.596520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:43.615883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01b448-d29d-4af5-ba61-275fe94e6814 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:43.735065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:43.736361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:43.737992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:43.738546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.739583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:43.742651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:43.838310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:43.941315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 857/1739] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:43 2022] POST /identity/v3/users => generated 284 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:43.941938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:43.978631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1dec1-bbed-4fed-ae7b-fce9c036530c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:43.992281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:43.993742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:43.995292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:43.996016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:43.997399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:44.003439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:44.034948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 883/1740] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:44.039222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:44.059119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7771cb8-9b49-4c79-b428-a67cf96b24a1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:44.079697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:44.082623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.086537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.088534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.090299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:44.097215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:44.116818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:44.119040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:44.151929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208c61c-a8b5-456c-8059-2ec30d5a8a94 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:44.166366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:44.168662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.170335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.171299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] PATH_INFO: `/v3/users/51b978eee8ea4ea3baa9168a5bb3a40c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.181536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b978eee8ea4ea3baa9168a5bb3a40c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:44.191322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 858/1741] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:44.196654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:44.198529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:44.231694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4c833-eeab-4630-98b3-136e1748f66f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:44.249610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:44.250543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.251467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.253306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] PATH_INFO: `/v3/projects/6ffab5e4434c4bad86f0a89a20b6a1d1/users/6ec6068eaf904a79bae43a55a1ce6200/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.293958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffab5e4434c4bad86f0a89a20b6a1d1, user_id=6ec6068eaf904a79bae43a55a1ce6200, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:44.302351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ba45e-fac2-48ec-831e-bbf73cc420f2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:44.390969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 859/1742] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:44 2022] PUT /identity/v3/projects/6ffab5e4434c4bad86f0a89a20b6a1d1/users/6ec6068eaf904a79bae43a55a1ce6200/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:44.392617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.394147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.394147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.438065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b978eee8ea4ea3baa9168a5bb3a40c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:44.438195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] MFA Rules not processed for user `6ec6068eaf904a79bae43a55a1ce6200`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:44.439901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:44.468819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b978eee8ea4ea3baa9168a5bb3a40c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:44.513478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b978eee8ea4ea3baa9168a5bb3a40c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:44.514309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b978eee8ea4ea3baa9168a5bb3a40c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:44.516093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-aeced5c9-e7e9-40c2-92d9-74ea3d55e8dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b978eee8ea4ea3baa9168a5bb3a40c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:44.600789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 860/1743] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:44.613962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:44.613962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:44.742881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:44.742881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029664a6-d327-429c-9747-8f46ae59ac77 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:44.743628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 884/1744] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:44 2022] DELETE /identity/v3/users/51b978eee8ea4ea3baa9168a5bb3a40c => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:44.743729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:44.743729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.743729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.743729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.744035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:44.744035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:44.753677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:44.754002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:44.754291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:44.843653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:44.845134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:45.072799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 885/1745] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:45.099263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:45.101353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:45.258144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce862ca1-2881-4864-bf1c-61e21626ba3b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:45.264582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:45.265342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:45.265797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:45.266086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:45.281786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:45.283147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 861/1746] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:44 2022] POST /identity/v3/projects => generated 367 bytes in 682 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:45.287763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:45.289056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:45.303327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 886/1747] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:45.494821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cc38c-363d-452b-b391-52c368e6f46a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:45.505128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:45.505128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:45.505128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:45.505128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:45.505470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:45.508041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:45.670752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:45.682040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 862/1748] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:45 2022] POST /identity/v3/users => generated 290 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:45.688882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3702db6-9b5c-43db-b234-8b9472ab3fd7 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:45.703818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:45.705023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:45.706328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:45.707258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:45.709374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:45.723644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:45.752725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:45.772957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 887/1749] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:45.775795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cac92-9405-4f17-bbb6-d53012389ac1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:45.782437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:45.783448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:45.786690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:45.787368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] PATH_INFO: `/v3/projects/55bd0abb237d40edac4e81596dedf3c3/users/00fcb66819c44202a343a340bb4e6ae4/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:45.806889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bd0abb237d40edac4e81596dedf3c3, user_id=00fcb66819c44202a343a340bb4e6ae4, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:45.809932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:45.930361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:45.931123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 863/1750] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:45 2022] PUT /identity/v3/projects/55bd0abb237d40edac4e81596dedf3c3/users/00fcb66819c44202a343a340bb4e6ae4/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:45.950876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee188956-53c2-4447-8e92-2989f3b37f48 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:46.140342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:46.141142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.141558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.142045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.143140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:46.145480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f036-d68e-468e-8dac-330a5bcae74d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:46.159379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 888/1751] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:46.163347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:46.190200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2140-7780-4c7a-b4d4-943bfc12c581 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:46.196923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:46.198670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.199175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.199690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] PATH_INFO: `/v3/projects/55bd0abb237d40edac4e81596dedf3c3/users/00fcb66819c44202a343a340bb4e6ae4/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.202191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bd0abb237d40edac4e81596dedf3c3, user_id=00fcb66819c44202a343a340bb4e6ae4, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:46.206610 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:46.222544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 864/1752] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:46 2022] PUT /identity/v3/projects/55bd0abb237d40edac4e81596dedf3c3/users/00fcb66819c44202a343a340bb4e6ae4/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:46.234030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.234576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.235059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.307159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] MFA Rules not processed for user `00fcb66819c44202a343a340bb4e6ae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:46.310075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:46.480949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 889/1753] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:46.543133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:46.545255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:46.594764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:46.596371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:46.672795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694378d-ceef-4578-9d77-a484790e0f5d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:46.676744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:46.679111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.679111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.679111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.689571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:46.691357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:46.696237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95a8b8-e2c3-44d6-b539-46f450d557e3 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:46.703075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:46.703957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.704252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.704671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] PATH_INFO: `/v3/projects/3044f3ab7eb347a38b84cea53e9e474c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.704886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 865/1754] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:46.706339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3044f3ab7eb347a38b84cea53e9e474c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:46.709055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:46.757476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:46.760279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:46.775985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1260db1-6543-476c-9500-c6cbd4953a97 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:46.781570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:46.782456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:46.782797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:46.783082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] PATH_INFO: `/v3/users/641ade3664db41dca1613ce8eac5d0e8` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:46.784300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=641ade3664db41dca1613ce8eac5d0e8)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:46.786473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:46.876935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3044f3ab7eb347a38b84cea53e9e474c', 'request_id': 'req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:46.885796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3044f3ab7eb347a38b84cea53e9e474c', 'request_id': 'req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.017515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641ade3664db41dca1613ce8eac5d0e8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.017515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641ade3664db41dca1613ce8eac5d0e8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.033326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3044f3ab7eb347a38b84cea53e9e474c', 'request_id': 'req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.042828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641ade3664db41dca1613ce8eac5d0e8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.052932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] The token cache is being invalidate because project 3044f3ab7eb347a38b84cea53e9e474c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:47.054512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.055587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641ade3664db41dca1613ce8eac5d0e8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.057235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.061864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 890/1755] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:46 2022] DELETE /identity/v3/projects/3044f3ab7eb347a38b84cea53e9e474c => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:47.063897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:47.069436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:47.138931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36637a7e-feaa-4498-9c37-cd9b2a0b27b4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:47.143097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:47.143838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:47.144239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:47.144876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:47.146933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:47.221372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:47.223118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:47.234768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 891/1756] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:47.235694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:47.237645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e6772906-820c-422b-ae85-eecaf2f5ac9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641ade3664db41dca1613ce8eac5d0e8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.246605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 866/1757] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:46 2022] DELETE /identity/v3/users/641ade3664db41dca1613ce8eac5d0e8 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:47.249089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:47.251035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:47.251339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:47.251628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:47.284655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:47.286584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:47.397887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932f1b3-7dec-428f-aa14-f7dfd752999a service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:47.438388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:47.456884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 867/1758] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:47.457917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:47.459727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:47.492149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:47.493408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:47.493795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:47.494226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] PATH_INFO: `/v3/users/32418947ac8442fcb4714f28afca8bd1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:47.495759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=32418947ac8442fcb4714f28afca8bd1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:47.498058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:47.566424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef08812-eb31-45f7-b7fb-d04ad0927c51 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:47.576149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:47.576149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:47.576149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:47.576149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:47.588533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:47.590485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:47.609091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 868/1759] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:47.748087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32418947ac8442fcb4714f28afca8bd1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.759825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32418947ac8442fcb4714f28afca8bd1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.779978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32418947ac8442fcb4714f28afca8bd1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.780802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32418947ac8442fcb4714f28afca8bd1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.781444 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32418947ac8442fcb4714f28afca8bd1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:47.891004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:47.896964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 892/1760] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:47 2022] DELETE /identity/v3/users/32418947ac8442fcb4714f28afca8bd1 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:48.315299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:48.317348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:48.354118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fb60c-f6ee-45be-a33f-fe09a42c5e41 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:48.433943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:48.434818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:48.435107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:48.435387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] PATH_INFO: `/v3/projects/dc9bd5454a8948a28ce05392ff2e13e0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:48.436622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9bd5454a8948a28ce05392ff2e13e0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:48.438555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:48.485517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:48.486999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:48.500205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e475ac0-4289-45b5-b13c-2981a243ccfc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:48.505370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:48.507157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:48.507446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:48.507725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] PATH_INFO: `/v3/projects/a11324889c744409a2f70afe872e352f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:48.509056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11324889c744409a2f70afe872e352f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:48.511004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:48.575412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9bd5454a8948a28ce05392ff2e13e0', 'request_id': 'req-15bdeb2b-33a3-4740-8e96-6ae326554672'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.583021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9bd5454a8948a28ce05392ff2e13e0', 'request_id': 'req-15bdeb2b-33a3-4740-8e96-6ae326554672'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.626380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11324889c744409a2f70afe872e352f', 'request_id': 'req-b5b67124-f3a1-4062-b2fc-179fe368e2f5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.633642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11324889c744409a2f70afe872e352f', 'request_id': 'req-b5b67124-f3a1-4062-b2fc-179fe368e2f5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.717692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9bd5454a8948a28ce05392ff2e13e0', 'request_id': 'req-15bdeb2b-33a3-4740-8e96-6ae326554672'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.729943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] The token cache is being invalidate because project dc9bd5454a8948a28ce05392ff2e13e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:48.730251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.730927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.732783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 869/1761] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:48 2022] DELETE /identity/v3/projects/dc9bd5454a8948a28ce05392ff2e13e0 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:48.736648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:48.738056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:48.786430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11324889c744409a2f70afe872e352f', 'request_id': 'req-b5b67124-f3a1-4062-b2fc-179fe368e2f5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.809961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdeb2b-33a3-4740-8e96-6ae326554672 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:48.819314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] The token cache is being invalidate because project a11324889c744409a2f70afe872e352f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:48.819619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.828816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:48.830893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 893/1762] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:48 2022] DELETE /identity/v3/projects/a11324889c744409a2f70afe872e352f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:48.834710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:48.836435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:48.883962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:48.885448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:48.885819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:48.886178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] PATH_INFO: `/v3/users/0bd8c4b14d6e4105861d3dd47e000034` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:48.887827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd8c4b14d6e4105861d3dd47e000034)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:48.890132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:48.924470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b67124-f3a1-4062-b2fc-179fe368e2f5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:48.932296 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:48.933406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:48.933771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:48.934291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] PATH_INFO: `/v3/users/0bc33adeb5f547b0bc8ad804714f0fff` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:48.937545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc33adeb5f547b0bc8ad804714f0fff)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:48.946908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:49.200450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd8c4b14d6e4105861d3dd47e000034'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.201090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd8c4b14d6e4105861d3dd47e000034'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.215503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd8c4b14d6e4105861d3dd47e000034'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.222966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd8c4b14d6e4105861d3dd47e000034'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.235354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc33adeb5f547b0bc8ad804714f0fff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.245882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc33adeb5f547b0bc8ad804714f0fff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.261995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc33adeb5f547b0bc8ad804714f0fff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.262805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc33adeb5f547b0bc8ad804714f0fff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.263561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc33adeb5f547b0bc8ad804714f0fff'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.306356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd8c4b14d6e4105861d3dd47e000034'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:49.313738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 870/1763] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:48 2022] DELETE /identity/v3/users/0bd8c4b14d6e4105861d3dd47e000034 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:49.406924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 894/1764] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:48 2022] DELETE /identity/v3/users/0bc33adeb5f547b0bc8ad804714f0fff => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:49.908416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:49.911069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:49.983690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf29fa-6a7f-403b-9f7a-198479717d47 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.006692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.008485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:50.064624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.065519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.065810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.066092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] PATH_INFO: `/v3/projects/301ab67ea9a44d808ec358604b529cd5` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.067312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=301ab67ea9a44d808ec358604b529cd5)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.069392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.094815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58870680-9267-42ea-b4d3-6b763cb3ff72 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.101084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.102055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.102873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.103354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] PATH_INFO: `/v3/projects/dd5cddf6345149f0ab42a206bf9bc848` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.106534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5cddf6345149f0ab42a206bf9bc848)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.110145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.221376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301ab67ea9a44d808ec358604b529cd5', 'request_id': 'req-128ddf79-7276-4c0e-bb54-44147379c023'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.240438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301ab67ea9a44d808ec358604b529cd5', 'request_id': 'req-128ddf79-7276-4c0e-bb54-44147379c023'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.242102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5cddf6345149f0ab42a206bf9bc848', 'request_id': 'req-05b13111-994c-46bc-8516-0eac097d1dae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.251803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5cddf6345149f0ab42a206bf9bc848', 'request_id': 'req-05b13111-994c-46bc-8516-0eac097d1dae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.340823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301ab67ea9a44d808ec358604b529cd5', 'request_id': 'req-128ddf79-7276-4c0e-bb54-44147379c023'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.354153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] The token cache is being invalidate because project 301ab67ea9a44d808ec358604b529cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:50.354723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.355382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128ddf79-7276-4c0e-bb54-44147379c023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.357053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 871/1765] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:49 2022] DELETE /identity/v3/projects/301ab67ea9a44d808ec358604b529cd5 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:50.361656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43531b-3297-40e7-a80b-5b5fba259077 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.361958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43531b-3297-40e7-a80b-5b5fba259077 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.362245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43531b-3297-40e7-a80b-5b5fba259077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.381095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-de43531b-3297-40e7-a80b-5b5fba259077 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:50.382405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-de43531b-3297-40e7-a80b-5b5fba259077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:50.454562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 872/1766] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:50.463493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.463799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.464209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.465924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5cddf6345149f0ab42a206bf9bc848', 'request_id': 'req-05b13111-994c-46bc-8516-0eac097d1dae'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.480677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] The token cache is being invalidate because project dd5cddf6345149f0ab42a206bf9bc848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:50.480986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.481806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05b13111-994c-46bc-8516-0eac097d1dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:50.484050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 895/1767] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:50 2022] DELETE /identity/v3/projects/dd5cddf6345149f0ab42a206bf9bc848 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:50.493818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:50.495069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:50.497181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274f-1c44-4baf-8504-621950d99f33 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.497603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274f-1c44-4baf-8504-621950d99f33 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.498009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d274f-1c44-4baf-8504-621950d99f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.504763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 873/1768] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:50.514049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.524591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57928429-2a13-430e-a18e-2fd2ec5fe078 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.530611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.531301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.531570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.531849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.532670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:50.532970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.535098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-368d274f-1c44-4baf-8504-621950d99f33 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:50.535337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.536501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-368d274f-1c44-4baf-8504-621950d99f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:50.550586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 874/1769] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:25:50.555874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.559954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 896/1770] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:50.569688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.569980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.570264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.584295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c7b24-697b-4857-93d4-680dbb09eae3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.591011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.591011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.591011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.591472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.592727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.594841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.610841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:50.617023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:50.632084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 897/1771] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:50.633108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.643381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f05d8-4f17-468d-aea5-f05134a26b40 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.649546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.650230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.650530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.650846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.651642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:50.652144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.656077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.671156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 898/1772] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:50.675620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.687549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d5923-a891-4fc9-bc7e-8d478feee880 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.706409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.707131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.707421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.707704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.708628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.725398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.733482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 875/1773] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:25:50.734415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.837913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966da4a9-cfc6-47d3-a820-47de1f97d4f1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.855506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.857754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.857754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.857754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.858222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.862507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:50.862942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 899/1774] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/projects => generated 393 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:50.866960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:50.941244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f42b8d-bd52-4199-a92b-5eec3b78f7de admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:50.946919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:50.947612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:50.947896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:50.948811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:50.950264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:50.952619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.005206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 876/1775] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/users => generated 291 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:51.008890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.019155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e688d-24d6-42ee-a80d-01ffc85adc43 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.024781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.025491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.025779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.026058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.026915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.028514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.039285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 877/1776] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:51.043063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.052788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970489b2-2278-4d53-b65f-ed7d25ff345d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.057480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.058126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.058467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.058756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.059481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.061188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.073073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 900/1777] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:50 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:51.075512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 878/1778] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:25:51.089364 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.089914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.101094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19499e3-059d-4e15-b62a-52a28da2f263 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.102684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771541-d1e5-4ba2-bab8-7516a55eb9f4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.107157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.107874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.108174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.108500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.109189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.109310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.110397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.110888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.110992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f4a34-31d8-404f-8b17-c9c5ad968479 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.111442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] PATH_INFO: `/v3/projects/9dd9f27ce1f143e5a33dd590a6dfd432/users/7a488dc736b44a56b293a43d48d0484f/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.124722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 901/1779] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:51.126776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674435-d70c-4064-a2e1-a060f3c0a238 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.127244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674435-d70c-4064-a2e1-a060f3c0a238 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.127788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55674435-d70c-4064-a2e1-a060f3c0a238 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.128764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd9f27ce1f143e5a33dd590a6dfd432, user_id=7a488dc736b44a56b293a43d48d0484f, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.130113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 902/1780] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:25:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:25:51.137235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674435-d70c-4064-a2e1-a060f3c0a238 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.137371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d634cc5-336a-499f-a2bf-42bf52b43fd2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.144571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55674435-d70c-4064-a2e1-a060f3c0a238 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.149701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.150647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.150647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.150983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.151718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.153724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0165-2e2c-4bb7-819b-1fe52d04abe8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.164896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 903/1781] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:51.166628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.166628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.166900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.189998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:51.191423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:51.261986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 904/1782] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:51.262745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.274890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a994bb-61ef-45a5-bacd-458c291ec1ca None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.295672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 879/1783] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:51 2022] PUT /identity/v3/projects/9dd9f27ce1f143e5a33dd590a6dfd432/users/7a488dc736b44a56b293a43d48d0484f/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:51.297112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.298114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.298573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.346866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] MFA Rules not processed for user `7a488dc736b44a56b293a43d48d0484f`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:51.349642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:51.422713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.423716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.424060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.425898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] PATH_INFO: `/v3/projects/c24ddd5595fd4bd69ab5ac87c3460337/users/21cb615b18f84f1f8b3c82f3fc49dd04/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.446776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24ddd5595fd4bd69ab5ac87c3460337, user_id=21cb615b18f84f1f8b3c82f3fc49dd04, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.449682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.457649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 880/1784] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:51.465418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.480914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5520e26-0d09-4029-bfb0-cada99eec930 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.487406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.488080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.488442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.488731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.489525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.492869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5acf-47e4-4a65-b92b-67af7399b843 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.598116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 905/1785] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:51 2022] PUT /identity/v3/projects/c24ddd5595fd4bd69ab5ac87c3460337/users/21cb615b18f84f1f8b3c82f3fc49dd04/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:51.599235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.600918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:51.634046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 881/1786] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:51.638587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.638892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.639180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.702672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] MFA Rules not processed for user `21cb615b18f84f1f8b3c82f3fc49dd04`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:51.703969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:51.723182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a05f0d-02a6-4ecd-b569-a0d80b9d004c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.791484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 882/1787] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:51.791484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.816793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.816793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.816793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.816793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.822195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.823816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.836528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 906/1788] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:51.837367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:51.868897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf3404-b21f-4087-af80-62b2469df08c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.874609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.875108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.875326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.875637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.876395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.878379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:51.910020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d39f44a-2ae3-4c65-90da-c4d1156a7baa service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:51.916090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:51.916810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:51.917116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:51.917389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:51.918105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:51.920020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.016186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 883/1789] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:52.017208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.028950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34aa61f-1180-4c3b-a267-b0ed728867d5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.034894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.035779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.036834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.037340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.039009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.040971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.058943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 884/1790] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:52.060147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.065714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 907/1791] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:51 2022] POST /identity/v3/projects => generated 391 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:52.071667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.077073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b90608-2cca-4143-9b77-87eb83476620 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.171692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.172705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.194618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.195013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] PATH_INFO: `/v3/projects/7176c86cef6047b796c81b0f4d94d8d1/users/f4831315d0504b2d82c6c246128f4b94/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.223400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69040ed3-be8f-43c3-a73d-95d48be78660 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.224985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7176c86cef6047b796c81b0f4d94d8d1, user_id=f4831315d0504b2d82c6c246128f4b94, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.227734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.230837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.232045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.232519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.232874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.233751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.235904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.309431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 885/1792] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:52 2022] PUT /identity/v3/projects/7176c86cef6047b796c81b0f4d94d8d1/users/f4831315d0504b2d82c6c246128f4b94/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:52.311095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.327240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 908/1793] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:52 2022] POST /identity/v3/users => generated 302 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:52.336828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.358853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dede1a-e405-455a-9162-79f18b806045 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.386999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:52.410473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059eff61-946a-4a45-bef5-d096beb311e9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.416835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.418036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.418541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.419580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.420688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.422824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.440344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 886/1794] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:52.441706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.456124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd3409-813b-4612-83d6-e65e3b34ca06 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.457120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.457953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.460564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.460564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.460816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.462544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.462789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.463444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.463734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.464013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] PATH_INFO: `/v3/projects/7176c86cef6047b796c81b0f4d94d8d1/users/f4831315d0504b2d82c6c246128f4b94/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.466339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7176c86cef6047b796c81b0f4d94d8d1, user_id=f4831315d0504b2d82c6c246128f4b94, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.469098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed0817-b61c-459d-a46d-5f5bdc6bd3dc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.474336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 909/1795] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:52.478708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.481298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 887/1796] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:52 2022] PUT /identity/v3/projects/7176c86cef6047b796c81b0f4d94d8d1/users/f4831315d0504b2d82c6c246128f4b94/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:52.489378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc6118-bde8-4402-8f50-8a8ccecb4902 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.491935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.492236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.492549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.542805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] MFA Rules not processed for user `f4831315d0504b2d82c6c246128f4b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:52.546912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:52.585722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.586823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.587171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.587460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] PATH_INFO: `/v3/projects/a7bc39e404bb4937907019db8a817c76/users/e5c73b9694c142ad8c0ff3593dab6369/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.606754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bc39e404bb4937907019db8a817c76, user_id=e5c73b9694c142ad8c0ff3593dab6369, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.611042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.654542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:52.672071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 888/1797] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:52.685618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.687145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:52.707670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 910/1798] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:52 2022] PUT /identity/v3/projects/a7bc39e404bb4937907019db8a817c76/users/e5c73b9694c142ad8c0ff3593dab6369/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:52.714402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.776138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed96ab-3b86-4ec9-b104-d8964bda5c2d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.822889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c3341-3e5e-4ac3-b6e8-59482c493fbd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.830891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.831593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.831886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.832176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.833122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.834836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.847620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 911/1799] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:52.853506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:52.867323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41f874-5728-4d3a-b4df-fbb52daa7a85 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:52.868531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.869428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.869907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.870329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.882245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:52.882245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.882245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.882245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] PATH_INFO: `/v3/projects/a7bc39e404bb4937907019db8a817c76/users/e5c73b9694c142ad8c0ff3593dab6369/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.882654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bc39e404bb4937907019db8a817c76, user_id=e5c73b9694c142ad8c0ff3593dab6369, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:52.882654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d904b6-48f8-46b9-bf7d-9d751b51b9d7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:52.896735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 912/1800] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:52 2022] PUT /identity/v3/projects/a7bc39e404bb4937907019db8a817c76/users/e5c73b9694c142ad8c0ff3593dab6369/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:52.908680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:52.909121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:52.909525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:52.972934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] MFA Rules not processed for user `e5c73b9694c142ad8c0ff3593dab6369`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:52.974179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:53.006423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:53.012732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:53.035510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 889/1801] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:53.104047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 913/1802] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:53.261333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:53.270056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2fb912-7471-4bd7-8bd8-37aeb3261b6b service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:53.332869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:53.332869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:53.332869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:53.332869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:53.339921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:53.341928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:53.354903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 890/1803] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:53.470464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:53.472027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:53.478332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:53.480336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:53.495909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d8b79-221a-400d-9620-ed06f0718a96 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:53.500911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:53.501624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:53.501916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:53.502210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:53.575580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:53.575945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface4df-9365-4b51-822b-bc7da6f924ed None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:53.577256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00051f21-a315-4194-a652-fee971e1bf75 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:53.581258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:53.587823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:53.587823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:53.587823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] PATH_INFO: `/v3/users/6ec6068eaf904a79bae43a55a1ce6200` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:53.587823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec6068eaf904a79bae43a55a1ce6200)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:53.588281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:53.589834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 891/1804] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:53.839799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec6068eaf904a79bae43a55a1ce6200'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:53.848498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec6068eaf904a79bae43a55a1ce6200'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:53.865339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec6068eaf904a79bae43a55a1ce6200'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:53.866363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec6068eaf904a79bae43a55a1ce6200'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:53.867141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec6068eaf904a79bae43a55a1ce6200'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:53.950742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 914/1805] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:53 2022] DELETE /identity/v3/users/6ec6068eaf904a79bae43a55a1ce6200 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:53.952023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:53.952338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:53.955656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:53.971531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:53.972899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:54.042390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 892/1806] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:54.054276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:54.056241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:54.143171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364bf09-a92f-4a4b-af0f-93221ea9e088 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:54.166274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:54.169295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:54.169611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:54.169895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:54.196945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:54.198532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:54.215206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 915/1807] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:54.815582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:54.817222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:54.830194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341306ec-6ee3-4633-94e3-58d0519aef69 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:54.835557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:54.836632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:54.836940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:54.837205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] PATH_INFO: `/v3/projects/6ffab5e4434c4bad86f0a89a20b6a1d1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:54.838462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffab5e4434c4bad86f0a89a20b6a1d1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:54.840481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:54.993395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffab5e4434c4bad86f0a89a20b6a1d1', 'request_id': 'req-85b97887-feea-4762-bbfe-9f024107a77d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:54.999648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffab5e4434c4bad86f0a89a20b6a1d1', 'request_id': 'req-85b97887-feea-4762-bbfe-9f024107a77d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.147062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffab5e4434c4bad86f0a89a20b6a1d1', 'request_id': 'req-85b97887-feea-4762-bbfe-9f024107a77d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.158337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] The token cache is being invalidate because project 6ffab5e4434c4bad86f0a89a20b6a1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:55.160386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.160386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.161159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 893/1808] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:54 2022] DELETE /identity/v3/projects/6ffab5e4434c4bad86f0a89a20b6a1d1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:55.165073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:55.175046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13632c57-b5ed-483a-bc3d-30e0bbffbd3a service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:55.233626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:55.234520 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:55.234863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:55.235144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] PATH_INFO: `/v3/users/00fcb66819c44202a343a340bb4e6ae4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:55.236380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fcb66819c44202a343a340bb4e6ae4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:55.238325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:55.446943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:55.454009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:55.530597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97887-feea-4762-bbfe-9f024107a77d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:55.535732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4831315d0504b2d82c6c246128f4b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7176c86cef6047b796c81b0f4d94d8d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:55.536540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:55.537414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:55.537796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:55.538834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:55.539245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:55.541158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:55.562190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 894/1809] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:25:55.566190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:55.577190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29bf3c-efcb-45c9-88e3-9144c8bd6530 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:55.582688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4831315d0504b2d82c6c246128f4b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7176c86cef6047b796c81b0f4d94d8d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:55.583992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:55.585015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:55.585411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:55.585818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fcb66819c44202a343a340bb4e6ae4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.586259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:55.588223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:55.593848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fcb66819c44202a343a340bb4e6ae4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.608053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fcb66819c44202a343a340bb4e6ae4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.619050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fcb66819c44202a343a340bb4e6ae4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.619050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fcb66819c44202a343a340bb4e6ae4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:55.718957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 895/1810] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:55 2022] POST /identity/v3/projects => generated 353 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:25:55.722907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:55.725354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:55.734909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 916/1811] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:55 2022] DELETE /identity/v3/users/00fcb66819c44202a343a340bb4e6ae4 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:55.792344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b9ff-a396-4a66-9fcb-ca6f2deea6f0 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:55.855603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4831315d0504b2d82c6c246128f4b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7176c86cef6047b796c81b0f4d94d8d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:55.856296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:55.856636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:55.856957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:55.857822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:55.858202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:55.859881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:55.876483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 896/1812] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:25:55.918303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:55.930080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b7e53-95aa-4608-8536-0fe9bde369f0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:55.934748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4831315d0504b2d82c6c246128f4b94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7176c86cef6047b796c81b0f4d94d8d1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:55.938686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:55.938686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:55.938686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] PATH_INFO: `/v3/projects/6753e1f0f96546d68daebbe6e7975a49` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:55.938686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6753e1f0f96546d68daebbe6e7975a49)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:55.939390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:56.003659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:56.065065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6753e1f0f96546d68daebbe6e7975a49', 'request_id': 'req-01fd753c-2ffb-410e-a69c-7c60a44c2385'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.070954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6753e1f0f96546d68daebbe6e7975a49', 'request_id': 'req-01fd753c-2ffb-410e-a69c-7c60a44c2385'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.169026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6753e1f0f96546d68daebbe6e7975a49', 'request_id': 'req-01fd753c-2ffb-410e-a69c-7c60a44c2385'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.181333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] The token cache is being invalidate because project 6753e1f0f96546d68daebbe6e7975a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:56.181626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.182288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-01fd753c-2ffb-410e-a69c-7c60a44c2385 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.183921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 917/1813] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:55 2022] DELETE /identity/v3/projects/6753e1f0f96546d68daebbe6e7975a49 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:56.257524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:56.259787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:56.341564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66090f2-9040-483e-a6fd-0050e0ea3376 tempest-VolumeQuotaClassesTest-1850379615 tempest-VolumeQuotaClassesTest-1850379615-project-admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:56.357569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:56.398567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:56.399430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.399717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.399992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] PATH_INFO: `/v3/projects/55bd0abb237d40edac4e81596dedf3c3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.401214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55bd0abb237d40edac4e81596dedf3c3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:56.403170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:56.500401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55bd0abb237d40edac4e81596dedf3c3', 'request_id': 'req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.507298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55bd0abb237d40edac4e81596dedf3c3', 'request_id': 'req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.620539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55bd0abb237d40edac4e81596dedf3c3', 'request_id': 'req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.634104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] The token cache is being invalidate because project 55bd0abb237d40edac4e81596dedf3c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:56.634104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.634684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-480d0a0f-67c3-4efa-aa04-e7f31d4e6b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:56.639865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 897/1814] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:56 2022] DELETE /identity/v3/projects/55bd0abb237d40edac4e81596dedf3c3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:56.641567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.641869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.642156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.666073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:56.667483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:56.751438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 918/1815] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:56.768125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.768469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.768763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.793579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:56.794939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:56.805148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 898/1816] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:56.809358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:56.819855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993750da-8e80-4dab-9327-8b6d85e53ccb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:56.825287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:56.825971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.826261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.826543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.827292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:25:56.827610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:56.829293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:56.841884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 919/1817] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:25:56.846287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:56.856742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822a46d-db69-448c-aa99-61376b8163bc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:56.861572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:56.862243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.862541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.862819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.863576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:56.866127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:56.924467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:56.926233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:56.940064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c4e-abcf-434c-a78e-4105fddc0cb4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:56.944930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:56.945821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:56.946109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:56.946392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] PATH_INFO: `/v3/users/7a488dc736b44a56b293a43d48d0484f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:56.947686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a488dc736b44a56b293a43d48d0484f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:56.949604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:56.999326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 899/1818] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:56 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:57.003133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.075289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395e086-8ee8-452c-b041-d88ae93446d2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.079908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.080618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.081883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.081883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.081883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.084333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.189682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a488dc736b44a56b293a43d48d0484f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:57.196847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a488dc736b44a56b293a43d48d0484f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:57.209993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a488dc736b44a56b293a43d48d0484f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:57.210737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a488dc736b44a56b293a43d48d0484f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:57.211373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f7dd4547-35a6-43b6-9b06-6d7826d33880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a488dc736b44a56b293a43d48d0484f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:57.218590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 900/1819] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:57 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:25:57.219719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.221388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:57.234742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203eb6d-9fab-4ee4-a4ec-9c37749f3902 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.240011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.242573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.242573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.242573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.242573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.243739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.253855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 901/1820] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:57.258101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.267690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9893b8a-43c2-47c4-9f1f-9e7cc77f2f70 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.273142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.273827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.274119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.274406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.275122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.276782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.287740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 902/1821] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:25:57.291900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.301755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f2-67e3-4985-93d6-5580b88ec62e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.306666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.307540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.307828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.308107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] PATH_INFO: `/v3/projects/1496930e3c604adfa8948badd8c40948/users/8cc9156c1cc041dfbd1f0ddc5b19ce67/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.319772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 920/1822] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:56 2022] DELETE /identity/v3/users/7a488dc736b44a56b293a43d48d0484f => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:57.324432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.324768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.325113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.327171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1496930e3c604adfa8948badd8c40948, user_id=8cc9156c1cc041dfbd1f0ddc5b19ce67, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.330033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22ca90-09a9-4579-87b2-46c725094edf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.344324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:57.345563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:57.402048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 903/1823] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:57 2022] PUT /identity/v3/projects/1496930e3c604adfa8948badd8c40948/users/8cc9156c1cc041dfbd1f0ddc5b19ce67/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:57.410473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.410896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.411317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.418883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 921/1824] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:57.429094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.431157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:57.444127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] MFA Rules not processed for user `8cc9156c1cc041dfbd1f0ddc5b19ce67`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:57.445448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:57.496544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd3ddc-5c05-4069-9dad-0c54e916ad45 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.525075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 904/1825] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:25:57.529828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.594723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.595606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.595728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.595946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.624881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba5e79-de5c-4201-80b8-19e9b0718a38 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.629496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63314390-c3de-497b-b814-654145768a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.630216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314390-c3de-497b-b814-654145768a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.630505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314390-c3de-497b-b814-654145768a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.630795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63314390-c3de-497b-b814-654145768a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.631694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63314390-c3de-497b-b814-654145768a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.634273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63314390-c3de-497b-b814-654145768a6e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.668707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.670311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.680846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 922/1826] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:57.749458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 905/1827] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:25:57 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:57.763557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.786305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867f9ef-450a-402c-b781-cb47197171b0 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:57.866914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:57.871019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:57.871019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:57.871019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:57.871019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:57.872923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:57.972521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 923/1828] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:25:57 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:25:57.977340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63314390-c3de-497b-b814-654145768a6e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:57.994339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63314390-c3de-497b-b814-654145768a6e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.002657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.003383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.007765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.007765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.007765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.007765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.021961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 906/1829] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:58.026047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.037428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617a932-88cd-4a4d-b286-42860d055c09 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.041957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.042882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.043174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.043460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] PATH_INFO: `/v3/projects/cbd85649b57b4469ab8824449bfabf94/users/b6d13857007d44478fe1597b91f3e183/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.061819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd85649b57b4469ab8824449bfabf94, user_id=b6d13857007d44478fe1597b91f3e183, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.066296 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.200998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 924/1830] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:58 2022] PUT /identity/v3/projects/cbd85649b57b4469ab8824449bfabf94/users/b6d13857007d44478fe1597b91f3e183/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:58.204704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.215949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9636a4-af9c-4c68-8500-18ce2086da0c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.240213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.242262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.288632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.289296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.289530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.289894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.290695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.292371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.304486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 907/1831] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:25:58.308467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.319799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cb63b-a5e6-4cda-b8e5-4ffe917cfb82 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.324806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.325733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.326017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.326297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] PATH_INFO: `/v3/projects/cbd85649b57b4469ab8824449bfabf94/users/b6d13857007d44478fe1597b91f3e183/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.328844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd85649b57b4469ab8824449bfabf94, user_id=b6d13857007d44478fe1597b91f3e183, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.334733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3052c-ced3-4cc1-bea8-638b0913cdd7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.336722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658654c-ea63-4232-a881-abb8918aba76 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.345159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.346045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.346333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.354820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] PATH_INFO: `/v3/projects/9dd9f27ce1f143e5a33dd590a6dfd432` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.355231 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 908/1832] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:25:58 2022] PUT /identity/v3/projects/cbd85649b57b4469ab8824449bfabf94/users/b6d13857007d44478fe1597b91f3e183/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:25:58.355325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd9f27ce1f143e5a33dd590a6dfd432)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.355325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.358988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.359284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.359567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.387795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] MFA Rules not processed for user `b6d13857007d44478fe1597b91f3e183`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:58.389044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.452108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd9f27ce1f143e5a33dd590a6dfd432', 'request_id': 'req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.459653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 909/1833] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:58.461477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd9f27ce1f143e5a33dd590a6dfd432', 'request_id': 'req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.471684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.473092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.529199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcc8e5-f1ac-4693-9bbc-80545595328b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.545836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd9f27ce1f143e5a33dd590a6dfd432', 'request_id': 'req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.559676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] The token cache is being invalidate because project 9dd9f27ce1f143e5a33dd590a6dfd432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:58.562143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.562273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:58.562371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.562871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 925/1834] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:58 2022] DELETE /identity/v3/projects/9dd9f27ce1f143e5a33dd590a6dfd432 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:58.568442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.569987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.582636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.583829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.587250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.587552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.600049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.665793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e07f8-d162-4604-8ebb-be3dc15b2b8a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.671763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.672771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.679495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.679495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] PATH_INFO: `/v3/users/f4831315d0504b2d82c6c246128f4b94` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.679495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4831315d0504b2d82c6c246128f4b94)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.679495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.693438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:58.698690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.700187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.716524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 910/1835] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:25:58.747728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:58.749899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:58.778914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1d0f6-1418-490b-8fb4-534b1bc74a92 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:58.788552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:58.788552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:58.788552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:58.788552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] PATH_INFO: `/v3/users/8cc9156c1cc041dfbd1f0ddc5b19ce67` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:58.789050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc9156c1cc041dfbd1f0ddc5b19ce67)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:58.798925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:58.918302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4831315d0504b2d82c6c246128f4b94'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.926830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4831315d0504b2d82c6c246128f4b94'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.939781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4831315d0504b2d82c6c246128f4b94'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.940613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4831315d0504b2d82c6c246128f4b94'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:58.941237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cd64498-a016-41bc-8066-6ee634d7a472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4831315d0504b2d82c6c246128f4b94'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.028479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 926/1836] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:58 2022] DELETE /identity/v3/users/f4831315d0504b2d82c6c246128f4b94 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:59.041776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc9156c1cc041dfbd1f0ddc5b19ce67'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.042555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc9156c1cc041dfbd1f0ddc5b19ce67'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.059250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc9156c1cc041dfbd1f0ddc5b19ce67'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.068176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc9156c1cc041dfbd1f0ddc5b19ce67'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.134689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc9156c1cc041dfbd1f0ddc5b19ce67'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.143529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 911/1837] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:25:58 2022] DELETE /identity/v3/users/8cc9156c1cc041dfbd1f0ddc5b19ce67 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:59.148177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:59.148575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:59.148934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:59.179424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:59.180703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:59.268955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 927/1838] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:59.279432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:59.282361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:59.360890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a89aa-d4dd-4160-a799-0ab09d11e775 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:59.365195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:59.365861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:59.366264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:59.366606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:59.376886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:59.378516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b33c05-b231-4dc8-8e77-b38ac451c72c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:59.392879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 912/1839] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:25:59.500327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:59.643863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:59.645369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:59.660549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9d5f0-628b-45ad-8d0b-d754fa1afe7a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:25:59.666807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:25:59.670272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:59.670272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:59.670272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] PATH_INFO: `/v3/projects/7176c86cef6047b796c81b0f4d94d8d1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:59.670272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7176c86cef6047b796c81b0f4d94d8d1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:25:59.671759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:25:59.780505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7176c86cef6047b796c81b0f4d94d8d1', 'request_id': 'req-21c87c33-edbb-4ca2-b701-1731b3edfe7d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.787894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7176c86cef6047b796c81b0f4d94d8d1', 'request_id': 'req-21c87c33-edbb-4ca2-b701-1731b3edfe7d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.838335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:25:59.860063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7176c86cef6047b796c81b0f4d94d8d1', 'request_id': 'req-21c87c33-edbb-4ca2-b701-1731b3edfe7d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.872158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] The token cache is being invalidate because project 7176c86cef6047b796c81b0f4d94d8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:25:59.872558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.877650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:25:59.877650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 928/1840] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:59 2022] DELETE /identity/v3/projects/7176c86cef6047b796c81b0f4d94d8d1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:25:59.882661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6803-929b-4dcf-b9c1-4b7181ef183a None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:59.882935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6803-929b-4dcf-b9c1-4b7181ef183a None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:59.883257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc6803-929b-4dcf-b9c1-4b7181ef183a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:59.902132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3ddc6803-929b-4dcf-b9c1-4b7181ef183a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:59.903387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3ddc6803-929b-4dcf-b9c1-4b7181ef183a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:59.950411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:25:59.951898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:25:59.966033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 913/1841] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:25:59.987156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:25:59.987156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:25:59.987156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:25:59.995642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:25:59.996949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.005313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 914/1842] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:00.012037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.022584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b81050-2138-4f26-b771-4c29a1ca17e7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.027745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.028481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.028774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.029056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.029790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:00.030134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.031823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.036091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c87c33-edbb-4ca2-b701-1731b3edfe7d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.041241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.042153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.042547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.042836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] PATH_INFO: `/v3/projects/1496930e3c604adfa8948badd8c40948` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.044093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1496930e3c604adfa8948badd8c40948)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.044746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 915/1843] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:26:00.049336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.050019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.059849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01787a5e-a11f-4469-87c6-02c8f2fb91cd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.064603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.065268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.065559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.065843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.066542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.068604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.177794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1496930e3c604adfa8948badd8c40948', 'request_id': 'req-a679c62d-731e-429a-baff-e0cc3d43ced2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.183155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 916/1844] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:00 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:00.185322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1496930e3c604adfa8948badd8c40948', 'request_id': 'req-a679c62d-731e-429a-baff-e0cc3d43ced2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.187114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.252479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339eba9-45e5-4a2e-9455-3adf54029529 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.256054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1496930e3c604adfa8948badd8c40948', 'request_id': 'req-a679c62d-731e-429a-baff-e0cc3d43ced2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.257995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.258736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.259088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.259371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.260131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.262194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.284005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] The token cache is being invalidate because project 1496930e3c604adfa8948badd8c40948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:00.284497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.285048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.287350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 929/1845] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:25:59 2022] DELETE /identity/v3/projects/1496930e3c604adfa8948badd8c40948 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:00.291406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.293903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.332003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 917/1846] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:00 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:00.336145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.338285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.353410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9249fc-abfb-4942-9c7e-1d34f5671fb5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.363846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.363846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.363846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.363846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.365568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.367643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.379500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 918/1847] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:00.384290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.391587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679c62d-731e-429a-baff-e0cc3d43ced2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.395710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49623e48-6a0f-4879-8605-2bc277d8892d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.398677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.399806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.400270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.400720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] PATH_INFO: `/v3/users/b6d13857007d44478fe1597b91f3e183` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.401450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.402177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.402479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.402751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.403314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d13857007d44478fe1597b91f3e183)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.403501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.405369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.406541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.417440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 919/1848] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:00.425515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.441174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc194f4-de96-4a52-b448-34883fca7cec admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.449037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.450124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.450570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.450924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] PATH_INFO: `/v3/projects/201bb98c1d75488398914a50071083f5/users/e79535ffccbd45438010b239843636be/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.470702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201bb98c1d75488398914a50071083f5, user_id=e79535ffccbd45438010b239843636be, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.473127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d9346-4f56-4143-a972-92b231d265e3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.617211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 920/1849] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:00 2022] PUT /identity/v3/projects/201bb98c1d75488398914a50071083f5/users/e79535ffccbd45438010b239843636be/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:00.629078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.629312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.629617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.670649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] MFA Rules not processed for user `e79535ffccbd45438010b239843636be`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:00.672144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.724683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d13857007d44478fe1597b91f3e183'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.736672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d13857007d44478fe1597b91f3e183'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.753416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d13857007d44478fe1597b91f3e183'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.754419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d13857007d44478fe1597b91f3e183'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.756484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d13857007d44478fe1597b91f3e183'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:00.765585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 921/1850] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:00.781720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.781720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.841426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1366fbc-b343-4791-93b8-10dc10c9fac5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:00.841426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:00.842078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:00.842179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:00.843122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:00.845187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:00.858207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:00.859826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:00.871321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 930/1851] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:00 2022] DELETE /identity/v3/users/b6d13857007d44478fe1597b91f3e183 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:00.871414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 922/1852] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:00.892527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:00.893949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:01.039091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42142c-ebd6-40ab-ace1-d810957c884f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:01.044716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:01.045625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:01.045919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:01.046205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] PATH_INFO: `/v3/users/e79535ffccbd45438010b239843636be` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:01.047526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e79535ffccbd45438010b239843636be)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:01.049878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:01.186281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:01.361441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e79535ffccbd45438010b239843636be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.369242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e79535ffccbd45438010b239843636be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.381064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e79535ffccbd45438010b239843636be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.381781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e79535ffccbd45438010b239843636be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.382386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b1332470-a279-40c9-8f1f-d6d2fb067f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e79535ffccbd45438010b239843636be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.430421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:01.438620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:01.495780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 931/1853] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:00 2022] DELETE /identity/v3/users/e79535ffccbd45438010b239843636be => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:01.497507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:01.497906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:01.498306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:01.557509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:01.566807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:01.673479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:01.701312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65b583-9227-4189-83ad-158ec3a25d7c service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:01.720338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 932/1854] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:01.731567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:01.733198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:01.778153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:01.779180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:01.779466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:01.780126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] PATH_INFO: `/v3/projects/cbd85649b57b4469ab8824449bfabf94` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:01.783326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd85649b57b4469ab8824449bfabf94)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:01.785329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:01.810350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756284-fe1c-4716-af47-001312b458d6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:01.814387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:01.815365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:01.815873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:01.816482 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:01.827286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:01.828802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff7af4-24f2-425c-a00e-73f5c2e941a4 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:01.844880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 933/1855] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:01.972222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd85649b57b4469ab8824449bfabf94', 'request_id': 'req-e9cd18e6-039b-4457-a70c-2884d1489733'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:01.982375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd85649b57b4469ab8824449bfabf94', 'request_id': 'req-e9cd18e6-039b-4457-a70c-2884d1489733'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.203611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd85649b57b4469ab8824449bfabf94', 'request_id': 'req-e9cd18e6-039b-4457-a70c-2884d1489733'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.216084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] The token cache is being invalidate because project cbd85649b57b4469ab8824449bfabf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:02.216424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.218075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e9cd18e6-039b-4457-a70c-2884d1489733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.219551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 923/1856] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:01 2022] DELETE /identity/v3/projects/cbd85649b57b4469ab8824449bfabf94 => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:02.226076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.226341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.226725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.247841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:02.249243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:02.329131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 934/1857] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:02.337614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.337911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.338196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.363150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:02.365372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:02.387286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 924/1858] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:02.391362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:02.401575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314850c-d9d2-4189-a4ac-d27838e88fdf None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:02.406408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:02.407108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.407403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.407688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.408487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:02.408827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:02.410503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:02.422286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 935/1859] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:02.426746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:02.436661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a167e-55b2-4f23-a722-ac3559620f68 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:02.441497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:02.442248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.443271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.443693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.444530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:02.446557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:02.494565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:02.496561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:02.519221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f9ffa-9098-462c-8098-f2bc098ed619 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:02.527774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:02.540558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.543869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.544165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] PATH_INFO: `/v3/users/21cb615b18f84f1f8b3c82f3fc49dd04` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.549490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21cb615b18f84f1f8b3c82f3fc49dd04)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:02.549490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:02.580900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 925/1860] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:02 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:02.586715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:02.586715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:02.657122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ef584-13cf-447f-b50e-d5ecc0438692 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:02.662948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:02.663887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:02.664262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:02.664636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] PATH_INFO: `/v3/projects/201bb98c1d75488398914a50071083f5` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:02.665979 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=201bb98c1d75488398914a50071083f5)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:02.668029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:02.808735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201bb98c1d75488398914a50071083f5', 'request_id': 'req-f47bad0b-de85-4832-bcfa-f0f44938a10d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.811718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21cb615b18f84f1f8b3c82f3fc49dd04'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.816157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201bb98c1d75488398914a50071083f5', 'request_id': 'req-f47bad0b-de85-4832-bcfa-f0f44938a10d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.821213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21cb615b18f84f1f8b3c82f3fc49dd04'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.840085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21cb615b18f84f1f8b3c82f3fc49dd04'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.841094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21cb615b18f84f1f8b3c82f3fc49dd04'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.841800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-be5789a7-c2e8-4498-b611-4511e3cdbdd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21cb615b18f84f1f8b3c82f3fc49dd04'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.940067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201bb98c1d75488398914a50071083f5', 'request_id': 'req-f47bad0b-de85-4832-bcfa-f0f44938a10d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.954642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] The token cache is being invalidate because project 201bb98c1d75488398914a50071083f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:02.955182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.956065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:02.957787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 926/1861] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:02 2022] DELETE /identity/v3/projects/201bb98c1d75488398914a50071083f5 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:02.959822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:02.961746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.033269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bad0b-de85-4832-bcfa-f0f44938a10d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.039026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.039722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.040013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.040338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.041072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.043230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.100747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 936/1862] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:02 2022] DELETE /identity/v3/users/21cb615b18f84f1f8b3c82f3fc49dd04 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:03.102089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9108d-3927-4ada-bb3d-efb7473c9339 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.102388 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9108d-3927-4ada-bb3d-efb7473c9339 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.102676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd9108d-3927-4ada-bb3d-efb7473c9339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.122587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7cd9108d-3927-4ada-bb3d-efb7473c9339 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:03.123814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7cd9108d-3927-4ada-bb3d-efb7473c9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.184260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 937/1863] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:03.185448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c14e2-8497-4a01-8489-d36695e28305 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.186376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c14e2-8497-4a01-8489-d36695e28305 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.186376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c14e2-8497-4a01-8489-d36695e28305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.204665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fc6c14e2-8497-4a01-8489-d36695e28305 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:03.205908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fc6c14e2-8497-4a01-8489-d36695e28305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.214260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 938/1864] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:03.215538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.215838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.216123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.223702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 927/1865] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:02 2022] POST /identity/v3/users => generated 284 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:03.224676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.228438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.238790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:03.241799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.254096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 939/1866] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:03.254096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.266395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef267af-6bd2-4128-9836-f104d033eb62 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.276747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.277539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.277833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.278120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.278884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.281059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.292992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 940/1867] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:03.293853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.301854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd08b27-cf09-4665-9522-74036c0b55ce admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.303882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636142bf-19af-45bb-9e9c-f48f45ef6c09 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.308816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.309478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.309768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.310050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.310793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:03.311102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.312748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.327302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 941/1868] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:03.327302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.338143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f211549-d53e-42c4-8bb3-809fad7b2f82 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.342970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.343646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.343952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.344415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.345179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.346884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.354598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.358724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.358724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.358724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.358985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 942/1869] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:03.359555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.366923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.368558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.371020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6bdcd-7fe2-4929-9ebb-31f995839b39 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.375916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.376692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.376924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.377211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.378194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.380546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 928/1870] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:03.380855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548476bf-37c0-4a21-b6f9-efc3062a00ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.381500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.391981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2186a4-669f-472c-bcfd-fa81ec96eb60 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.398547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.399702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.399988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.400271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] PATH_INFO: `/v3/projects/de220e7ac79146509e2e1f07bdff977e/users/a1f502b4b1514837b10c40191a159b58/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.417516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de220e7ac79146509e2e1f07bdff977e, user_id=a1f502b4b1514837b10c40191a159b58, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.420141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.577521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 929/1871] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:03 2022] PUT /identity/v3/projects/de220e7ac79146509e2e1f07bdff977e/users/a1f502b4b1514837b10c40191a159b58/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:03.578586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 943/1872] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/projects => generated 359 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:03.579215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.579311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.579311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.579311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.610382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] MFA Rules not processed for user `a1f502b4b1514837b10c40191a159b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:03.611711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:03.768954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29026-099b-4dac-82e9-c42be3b80feb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.769949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 944/1873] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:03.770024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.770024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.770024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.770024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.770287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.770287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.777841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:03.787515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd633e-c668-4fae-a371-a6928ab76d33 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:03.792218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:03.792911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:03.793198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:03.793478 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:03.794192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:03.796115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:03.932803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 930/1874] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/users => generated 280 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:03.991709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.003775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417aa65-f9e5-4b2c-8593-6cd1b23183d7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.010233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.011078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.011398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.011759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.012652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.014576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.038164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 931/1875] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:04.057534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.070436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55737858-41b9-4673-8590-1a3f542c67f0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.077694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.078404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.078696 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.078985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.079964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.081848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.096265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 932/1876] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:04.101724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.113940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85560406-9005-4b41-bad8-fe0807d88645 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.119071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.120006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.120291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.120609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] PATH_INFO: `/v3/projects/eb5cdb57f8d24daea2cac7306cf5d22d/users/1e88ba421af247d3a0d62c69453ba483/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.139455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5cdb57f8d24daea2cac7306cf5d22d, user_id=1e88ba421af247d3a0d62c69453ba483, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.142254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef929212-ff26-4b16-a52a-7dadcb114182 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.285854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 945/1877] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:03 2022] POST /identity/v3/projects => generated 367 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:04.289871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.387437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e9cfc-4e9e-42f5-a613-1617e3b508fa admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.405636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 933/1878] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:04 2022] PUT /identity/v3/projects/eb5cdb57f8d24daea2cac7306cf5d22d/users/1e88ba421af247d3a0d62c69453ba483/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:04.418673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.418989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.419113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.461644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.462977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.462977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.463190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.463898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.466029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.468783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] MFA Rules not processed for user `1e88ba421af247d3a0d62c69453ba483`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:04.470019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:04.575072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 934/1879] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:04 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:04.576276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.577894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:04.585623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 946/1880] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:04 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:04.591509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.598582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526480b2-ff16-4cf2-8bf4-2adee7d34795 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.604159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.605698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.606172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.607995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f6ce4-af75-4b90-a918-a96901835c4d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.614301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.616008 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.616330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.616630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.624064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] PATH_INFO: `/v3/projects/c24ddd5595fd4bd69ab5ac87c3460337` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.624487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.624487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.624699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:04.626558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24ddd5595fd4bd69ab5ac87c3460337)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.628933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.638819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 947/1881] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:04.642817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.666066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e672ec-115e-4fde-b604-da798893f623 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.673167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.674381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.674848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.675164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] PATH_INFO: `/v3/projects/834d98cfb7a2427baaef31654abd9774/users/6176c85a1e524d2cb8e80f096bc95f98/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.694809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834d98cfb7a2427baaef31654abd9774, user_id=6176c85a1e524d2cb8e80f096bc95f98, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.706114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:04.795416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 948/1882] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:04 2022] PUT /identity/v3/projects/834d98cfb7a2427baaef31654abd9774/users/6176c85a1e524d2cb8e80f096bc95f98/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:04.796638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.799222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:04.804470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24ddd5595fd4bd69ab5ac87c3460337', 'request_id': 'req-b684b562-4189-4b54-99ab-b52994427b78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:04.813413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24ddd5595fd4bd69ab5ac87c3460337', 'request_id': 'req-b684b562-4189-4b54-99ab-b52994427b78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:04.871773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9f0cf-5316-44e0-ba0c-a2009f808995 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:04.883981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:04.933338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:04.933995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:04.934281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:04.934562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:04.957222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24ddd5595fd4bd69ab5ac87c3460337', 'request_id': 'req-b684b562-4189-4b54-99ab-b52994427b78'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:04.969078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] The token cache is being invalidate because project c24ddd5595fd4bd69ab5ac87c3460337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:04.975957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:04.975957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:04.975957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 935/1883] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:04 2022] DELETE /identity/v3/projects/c24ddd5595fd4bd69ab5ac87c3460337 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:04.975957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:04.986792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:04.996702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:04.998271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:05.011945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 949/1884] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:04 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:05.011945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:05.067031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684b562-4189-4b54-99ab-b52994427b78 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:05.072623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:05.073321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.073778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.074068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.074826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:05.076551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:05.086908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d6692-5111-4dc2-9fb0-4b588aeaf0f0 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:05.088648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 936/1885] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:05.089520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:05.091212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:05.094148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:05.095307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.095608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.095888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] PATH_INFO: `/v3/users/e5c73b9694c142ad8c0ff3593dab6369` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.097185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c73b9694c142ad8c0ff3593dab6369)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:05.099998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:05.106689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bca58-50c5-44bd-870a-0dfc48e5b776 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:05.111925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:05.112863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.113154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.113441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] PATH_INFO: `/v3/users/1e88ba421af247d3a0d62c69453ba483` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.116005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e88ba421af247d3a0d62c69453ba483)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:05.118048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:05.395845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c73b9694c142ad8c0ff3593dab6369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.404135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c73b9694c142ad8c0ff3593dab6369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.419825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c73b9694c142ad8c0ff3593dab6369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.420624 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c73b9694c142ad8c0ff3593dab6369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.421534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c73b9694c142ad8c0ff3593dab6369'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.432234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e88ba421af247d3a0d62c69453ba483'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.433450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e88ba421af247d3a0d62c69453ba483'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.452524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e88ba421af247d3a0d62c69453ba483'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.463360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e88ba421af247d3a0d62c69453ba483'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.585921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 950/1886] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:05 2022] DELETE /identity/v3/users/e5c73b9694c142ad8c0ff3593dab6369 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:05.586970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:05.588559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:05.637162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2c0bd779-c625-428d-9162-6aaa7a91e880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e88ba421af247d3a0d62c69453ba483'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:05.648109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 937/1887] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:05 2022] DELETE /identity/v3/users/1e88ba421af247d3a0d62c69453ba483 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:05.656636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.656820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.657117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.690542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:05.692026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:05.699086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ed9c6-839f-422f-82ac-f8e2732b56da admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:05.772248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 938/1888] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:05.779365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:05.780939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:05.796893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:05.797872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.798167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.798511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] PATH_INFO: `/v3/projects/834d98cfb7a2427baaef31654abd9774/users/6176c85a1e524d2cb8e80f096bc95f98/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.800858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=834d98cfb7a2427baaef31654abd9774, user_id=6176c85a1e524d2cb8e80f096bc95f98, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:05.803972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d099-3ce1-4393-ade2-46ccc88876c8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:05.819865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 951/1889] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:05 2022] PUT /identity/v3/projects/834d98cfb7a2427baaef31654abd9774/users/6176c85a1e524d2cb8e80f096bc95f98/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:05.837502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.837802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.838096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:05.859545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f758e45-7048-445b-a3ee-7591ee0f7ae9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:05.914782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] MFA Rules not processed for user `6176c85a1e524d2cb8e80f096bc95f98`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:05.914782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:05.943357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:05.944158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:05.944910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:05.945211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:06.031659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:06.034185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:06.052858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 952/1890] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:06.054215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 939/1891] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:06.072818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:06.074705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:06.183371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4de9e4-d63c-4595-aae5-cc1aed0d159c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:06.187223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:06.187962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:06.188342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:06.188739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:06.200168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:06.202437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:06.213790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 953/1892] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:06.582423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:06.595790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe2678-bd3c-4ebe-acd7-b6929724cec4 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:06.596854 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:06.598385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:06.629738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:06.631577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:06.632561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:06.633099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:06.633598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:06.636487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:06.641415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df39ee5-e661-4a7d-bc54-ce7108c46e4a service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:06.655485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:06.656599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:06.657000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:06.657387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] PATH_INFO: `/v3/projects/eb5cdb57f8d24daea2cac7306cf5d22d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:06.658883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5cdb57f8d24daea2cac7306cf5d22d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:06.661105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:06.714209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:06.716407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:06.750831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 940/1893] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:06.851552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5cdb57f8d24daea2cac7306cf5d22d', 'request_id': 'req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:06.860845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5cdb57f8d24daea2cac7306cf5d22d', 'request_id': 'req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:06.875054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:06.890192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:06.891882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:06.919631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0747c-2f00-4d36-8763-653f6ebb4378 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:06.983520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:06.984465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:06.984756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:06.985039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] PATH_INFO: `/v3/projects/a7bc39e404bb4937907019db8a817c76` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:06.986369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bc39e404bb4937907019db8a817c76)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:06.988413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.019680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5cdb57f8d24daea2cac7306cf5d22d', 'request_id': 'req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.036853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] The token cache is being invalidate because project eb5cdb57f8d24daea2cac7306cf5d22d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:07.038622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.038765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b8abe78c-e7e0-49a2-8c0d-74d884ce7c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.040580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 954/1894] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:06 2022] DELETE /identity/v3/projects/eb5cdb57f8d24daea2cac7306cf5d22d => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:07.045382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e15f3-d92b-4fc9-a072-599496d3f35a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.045693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e15f3-d92b-4fc9-a072-599496d3f35a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.045976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e15f3-d92b-4fc9-a072-599496d3f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.085088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-661e15f3-d92b-4fc9-a072-599496d3f35a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:07.086337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-661e15f3-d92b-4fc9-a072-599496d3f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:07.096342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 955/1895] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:07.104528 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.104948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.105247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.125446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:07.126686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:07.135415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 956/1896] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:07.139533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.149212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fce23a-3128-4923-b388-0e6113734a2c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.154241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.155048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.155420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.155718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.156506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:07.156824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.158467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.172663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 957/1897] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:26:07.178301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.182318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bc39e404bb4937907019db8a817c76', 'request_id': 'req-3e1b1128-89c1-4c84-b244-6f30ef74327d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.188109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b872ec-adb1-43f1-93a6-7b594aaa28cf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.188607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bc39e404bb4937907019db8a817c76', 'request_id': 'req-3e1b1128-89c1-4c84-b244-6f30ef74327d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.249510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.250092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.250384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.250668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.251373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.253419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.322999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bc39e404bb4937907019db8a817c76', 'request_id': 'req-3e1b1128-89c1-4c84-b244-6f30ef74327d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.339296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] The token cache is being invalidate because project a7bc39e404bb4937907019db8a817c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:07.340561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.341704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3e1b1128-89c1-4c84-b244-6f30ef74327d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:07.344351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 941/1898] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:06 2022] DELETE /identity/v3/projects/a7bc39e404bb4937907019db8a817c76 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:07.352831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4651-66b0-4684-96be-9bc9cbd85409 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.358720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4651-66b0-4684-96be-9bc9cbd85409 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.359169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e4651-66b0-4684-96be-9bc9cbd85409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.363415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 958/1899] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:07.367459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.369266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:07.391261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c02e4651-66b0-4684-96be-9bc9cbd85409 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:07.396092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c02e4651-66b0-4684-96be-9bc9cbd85409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:07.483789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c329d-eb5a-4f8b-adca-80b08f6c3c47 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.491637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 942/1900] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:07.498770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.504516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.504720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.505466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.505608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.505739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.505822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.505904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.505904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.524725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:07.525999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:07.534981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 943/1901] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:07.541722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.552702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c34381-8409-4c9f-97e4-7c896442a721 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.558458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.559154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.559450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.559737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.560557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:07.561076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.563162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.573244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:07.581052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 944/1902] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:26:07.584652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.595632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c1c2b-de2b-4fa8-bd4e-8cac24b31bf9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.600741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.601434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.606174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.606174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.606174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.606174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.656439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 959/1903] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:07.660774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.676595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe1e69-4730-4747-9034-4e555714835a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.694106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.695160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.695525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.695885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.696821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.699017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.714093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 960/1904] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:07.718644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.730896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14b08a-0c74-440d-a6ef-40edb8da9a3f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.742791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 945/1905] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/projects => generated 409 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:07.747078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.747475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.748462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.748890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.749353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.750356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.752145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.764791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 961/1906] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:07.769751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:07.785342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d09273-9d53-417d-9872-2aa05d1d0853 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.937560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47786e-0db2-49f0-a59c-e06b517b89fe admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:07.940084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.941694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.941981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.942289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] PATH_INFO: `/v3/projects/95f6c62ac1a049339b4572db556a4e3b/users/14281b945e374b6fa6d610085e31f1b4/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.944824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:07.946212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:07.946606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:07.947013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:07.948026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.950130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770473d-0360-4c2f-829e-0bf344e74969 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:07.971977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f6c62ac1a049339b4572db556a4e3b, user_id=14281b945e374b6fa6d610085e31f1b4, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:07.975805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.115052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 962/1907] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:07 2022] PUT /identity/v3/projects/95f6c62ac1a049339b4572db556a4e3b/users/14281b945e374b6fa6d610085e31f1b4/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:08.115157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 946/1908] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:07 2022] POST /identity/v3/users => generated 305 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:08.119519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.127831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.130771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.130771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.186586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] MFA Rules not processed for user `14281b945e374b6fa6d610085e31f1b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:08.187870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:08.281506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4136fe9-919b-4d8e-ad06-bd897edebc47 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.296686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.305272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.305272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.305272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.305272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.305272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.338823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 963/1909] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:08.342666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.362064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73527d8-3972-4288-89c7-970618c8d299 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.370147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.371939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.372639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.373127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.374298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.376136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.384271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 947/1910] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:08.389307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.414189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c631c0-de68-4e58-9b0e-3d4d1abb7387 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.420244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.420527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:08.421103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.421526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.421911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.422793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 964/1911] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:08.426334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.428225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.430134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.468009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27efdfa-fd05-4f66-b28d-26f05c0ccf1a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.489683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.493202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.496656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.497405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] PATH_INFO: `/v3/projects/2ab4321ed4b24d2aa1fdcc565426b682/users/8579d20c2ebc4848a71096e14f5f1455/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.514120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 948/1912] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:08 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:08.517709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.553340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab4321ed4b24d2aa1fdcc565426b682, user_id=8579d20c2ebc4848a71096e14f5f1455, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.555784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebbace-7fa5-47df-a529-24bb33bf133d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.641504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25550-401a-49ef-88a8-71a2b4de4789 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.650286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.651292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.651643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.651989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.653036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.655417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.680055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 965/1913] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:08 2022] PUT /identity/v3/projects/2ab4321ed4b24d2aa1fdcc565426b682/users/8579d20c2ebc4848a71096e14f5f1455/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:08.690362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.692974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.693992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.740462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] MFA Rules not processed for user `8579d20c2ebc4848a71096e14f5f1455`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:08.741976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:08.752416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 949/1914] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:08 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:08.757368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.777746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8d19c-2e96-4177-b112-a72f9fd16ee5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:08.842137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:08.867049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 966/1915] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:08.871295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.927537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:08.928545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:08.929264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:08.930297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:08.931843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:08.935310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:08.973675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 950/1916] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:08.975187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:08.996447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c9839-691d-45c9-bf6c-35972105bdec admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.004361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.005967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.006368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.006858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] PATH_INFO: `/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a/users/78ac1cce0d5f4f38845c1a30fb3d36be/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.031600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d48564b9cba423f9ebfb3c5391ff03a, user_id=78ac1cce0d5f4f38845c1a30fb3d36be, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.037557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.050047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4b0c4-6d88-4e35-a396-34d8346a4ddd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.059583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.060288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.060590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.060876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.061595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.063554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.147472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 951/1917] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:08 2022] PUT /identity/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a/users/78ac1cce0d5f4f38845c1a30fb3d36be/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:09.151674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.202916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 967/1918] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:08 2022] POST /identity/v3/projects => generated 407 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:09.206792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.256049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179fc3c-352f-418b-ba0d-690267e1d50c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.314469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f103c-0dbd-419c-af49-05f0ee445bdd admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.322639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.323464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.323762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.324051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.325822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.330043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.350923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.352158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.352275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.352958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.353346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.355253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.407660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 952/1919] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:09.414345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.430838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52783e-0b5c-4937-aeef-a023c2043c70 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.437677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.438625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.438920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.439208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] PATH_INFO: `/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a/users/78ac1cce0d5f4f38845c1a30fb3d36be/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.441508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d48564b9cba423f9ebfb3c5391ff03a, user_id=78ac1cce0d5f4f38845c1a30fb3d36be, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.444976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518c329-ad40-4748-a142-86d341d993a1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.460528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 953/1920] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:09 2022] PUT /identity/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a/users/78ac1cce0d5f4f38845c1a30fb3d36be/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:09.470842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.474164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.474164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.480981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 968/1921] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:09 2022] POST /identity/v3/users => generated 310 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:09.485099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.499352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c66972-8933-4456-94b6-ce4a4bdd8c11 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.526649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] MFA Rules not processed for user `78ac1cce0d5f4f38845c1a30fb3d36be`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:09.530007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:09.587211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.588118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.589758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.589758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.590652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.592773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.608104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 969/1922] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:09.610075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.621680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ba456-e7b7-43b8-8d74-8fdbfe3c37ed admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.629192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.629373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.629575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.630419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] PATH_INFO: `/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951/users/3e5ed8131bfc4312a193922d0c4c4d49/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.643939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 954/1923] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:09.652483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eef6427bbe64c5fbfdd21ead3fd8951, user_id=3e5ed8131bfc4312a193922d0c4c4d49, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.655621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.700489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:09.730594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.732516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:09.753307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 970/1924] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:09 2022] PUT /identity/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951/users/3e5ed8131bfc4312a193922d0c4c4d49/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:09.761404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.831460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e5e34-d9ac-4a27-bdc8-89d435d0f720 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.873974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2f9a8-b263-48ba-909b-1b8e9bee367c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.881349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.891863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.891863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.891863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.891863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.891863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.910348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 971/1925] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:09.919398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:09.933914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.935323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e3cb6-ddf3-4588-ace7-6b3b83ca317e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:09.935788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.936080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.936592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.943719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:09.944780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.945150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.945442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] PATH_INFO: `/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951/users/3e5ed8131bfc4312a193922d0c4c4d49/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:09.947793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eef6427bbe64c5fbfdd21ead3fd8951, user_id=3e5ed8131bfc4312a193922d0c4c4d49, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:09.950567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87a55b-017c-4dba-a73c-ab29770813ad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:09.962498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 972/1926] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:09 2022] PUT /identity/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951/users/3e5ed8131bfc4312a193922d0c4c4d49/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:09.975015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:09.975569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:09.975853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:10.007511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:10.007732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] MFA Rules not processed for user `3e5ed8131bfc4312a193922d0c4c4d49`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:10.009000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:10.009258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:10.021178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 955/1927] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:10.136469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 973/1928] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:09 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:10.983482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:10.994831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5368ce2-a576-4141-8cf0-52cebbefbd49 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:11.064442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:11.065630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:11.066093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:11.066549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:11.077821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:11.079702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:11.102109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 956/1929] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:10 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:11.119005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:11.165519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:11.814400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:11.815978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:11.835679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5c15-97c1-4715-8cfb-7ba128fa25c2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:11.839590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:11.840468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:11.840751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:11.841113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:11.941109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:11.943422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2fcd-20b4-4dbe-92a9-5653a7f55cec service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:11.954418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 974/1930] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:11 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:13.345560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:13.408071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:14.330092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:14.402205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0fa20-773b-46f2-9416-189334605bb9 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:14.407796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:14.408966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:14.409255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:14.409531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] PATH_INFO: `/v3/users/14281b945e374b6fa6d610085e31f1b4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:14.410898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14281b945e374b6fa6d610085e31f1b4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:14.413324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:14.680351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14281b945e374b6fa6d610085e31f1b4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:14.680966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14281b945e374b6fa6d610085e31f1b4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:14.692989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14281b945e374b6fa6d610085e31f1b4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:14.699903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14281b945e374b6fa6d610085e31f1b4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:14.854441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14281b945e374b6fa6d610085e31f1b4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:14.861119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 957/1931] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:14 2022] DELETE /identity/v3/users/14281b945e374b6fa6d610085e31f1b4 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:14.867007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:14.867504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:14.867628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:14.885912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:14.887211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:14.949440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 975/1932] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:14.965356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:14.965356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:14.985618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:14.987158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:14.999192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3906407-289c-46e3-a49b-f1d86536f6ec None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:15.003795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:15.004709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:15.005021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:15.005304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] PATH_INFO: `/v3/users/a1f502b4b1514837b10c40191a159b58` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:15.006662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f502b4b1514837b10c40191a159b58)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:15.008667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:15.027027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722fc61-0717-4369-9415-f20ea6635f97 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:15.031646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:15.032463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:15.032817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:15.036602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:15.043890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:15.045383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3feca-e65c-47a2-af25-3347a0565f96 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:15.055965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 958/1933] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:15.260461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:15.308697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f502b4b1514837b10c40191a159b58'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.316277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f502b4b1514837b10c40191a159b58'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.330502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f502b4b1514837b10c40191a159b58'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.331258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f502b4b1514837b10c40191a159b58'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.331878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f502b4b1514837b10c40191a159b58'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.422937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 976/1934] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:14 2022] DELETE /identity/v3/users/a1f502b4b1514837b10c40191a159b58 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:15.429895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:15.430680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:15.430859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:15.470189 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:15.474952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:15.589737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 959/1935] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:15.600821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:15.602270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:15.635656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:15.639527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:15.662140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161b4bd-e940-4714-9a3b-e39c805d832b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:15.683039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:15.684167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:15.684724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:15.685111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] PATH_INFO: `/v3/projects/95f6c62ac1a049339b4572db556a4e3b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:15.686664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f6c62ac1a049339b4572db556a4e3b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:15.688799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:15.709899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8b2ce-01f9-4259-9ea2-7971303af9ef admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:15.714792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:15.715544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:15.715840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:15.716119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:15.726744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:15.728193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:15.739893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 977/1936] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:15.805360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f6c62ac1a049339b4572db556a4e3b', 'request_id': 'req-5d38b1de-effb-48c5-a59a-824268decf44'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.811825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f6c62ac1a049339b4572db556a4e3b', 'request_id': 'req-5d38b1de-effb-48c5-a59a-824268decf44'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.918502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f6c62ac1a049339b4572db556a4e3b', 'request_id': 'req-5d38b1de-effb-48c5-a59a-824268decf44'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.933121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] The token cache is being invalidate because project 95f6c62ac1a049339b4572db556a4e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:15.933565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.934447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:15.936133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 960/1937] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:15 2022] DELETE /identity/v3/projects/95f6c62ac1a049339b4572db556a4e3b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:15.940871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:15.951264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b628e-24b3-49f6-9fe5-7415283111cc service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:16.004211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:16.021830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:16.022735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:16.023025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:16.023307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] PATH_INFO: `/v3/users/78ac1cce0d5f4f38845c1a30fb3d36be` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:16.024609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ac1cce0d5f4f38845c1a30fb3d36be)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:16.026604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:16.341301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:16.342948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:16.356915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38b1de-effb-48c5-a59a-824268decf44 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:16.361978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:16.362870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:16.363299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:16.363588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] PATH_INFO: `/v3/projects/de220e7ac79146509e2e1f07bdff977e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:16.365862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=de220e7ac79146509e2e1f07bdff977e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:16.367823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:16.380416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ac1cce0d5f4f38845c1a30fb3d36be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.391016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ac1cce0d5f4f38845c1a30fb3d36be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.404216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ac1cce0d5f4f38845c1a30fb3d36be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.404839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ac1cce0d5f4f38845c1a30fb3d36be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.405493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ac1cce0d5f4f38845c1a30fb3d36be'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.510859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 978/1938] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:15 2022] DELETE /identity/v3/users/78ac1cce0d5f4f38845c1a30fb3d36be => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:16.512298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de220e7ac79146509e2e1f07bdff977e', 'request_id': 'req-e7be6b27-276c-4a60-9006-8ef32dbcfeed'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.519048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de220e7ac79146509e2e1f07bdff977e', 'request_id': 'req-e7be6b27-276c-4a60-9006-8ef32dbcfeed'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.603846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de220e7ac79146509e2e1f07bdff977e', 'request_id': 'req-e7be6b27-276c-4a60-9006-8ef32dbcfeed'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.617081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] The token cache is being invalidate because project de220e7ac79146509e2e1f07bdff977e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:16.617381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.618147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.619877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 961/1939] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:16 2022] DELETE /identity/v3/projects/de220e7ac79146509e2e1f07bdff977e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:16.623870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:16.625311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:16.689209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eed326-3730-461d-8009-f78f60f1f974 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:16.694257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:16.695150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:16.695446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:16.695730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] PATH_INFO: `/v3/users/6176c85a1e524d2cb8e80f096bc95f98` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:16.696976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6176c85a1e524d2cb8e80f096bc95f98)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:16.698975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:16.745679 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:16.914183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6176c85a1e524d2cb8e80f096bc95f98'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.923303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6176c85a1e524d2cb8e80f096bc95f98'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.935778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6176c85a1e524d2cb8e80f096bc95f98'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.937213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6176c85a1e524d2cb8e80f096bc95f98'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:16.937841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6176c85a1e524d2cb8e80f096bc95f98'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.089891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 979/1940] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:16 2022] DELETE /identity/v3/users/6176c85a1e524d2cb8e80f096bc95f98 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:17.090686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:17.092131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:17.176989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be6b27-276c-4a60-9006-8ef32dbcfeed admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:17.235747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:17.236664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:17.236953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:17.237251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] PATH_INFO: `/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:17.238440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d48564b9cba423f9ebfb3c5391ff03a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:17.241264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:17.382841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d48564b9cba423f9ebfb3c5391ff03a', 'request_id': 'req-13d2e562-73f3-419c-9995-8fcd5720a506'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.389133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d48564b9cba423f9ebfb3c5391ff03a', 'request_id': 'req-13d2e562-73f3-419c-9995-8fcd5720a506'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.459978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d48564b9cba423f9ebfb3c5391ff03a', 'request_id': 'req-13d2e562-73f3-419c-9995-8fcd5720a506'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.471063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] The token cache is being invalidate because project 2d48564b9cba423f9ebfb3c5391ff03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:17.471352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.471984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-13d2e562-73f3-419c-9995-8fcd5720a506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.473631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 962/1941] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:17 2022] DELETE /identity/v3/projects/2d48564b9cba423f9ebfb3c5391ff03a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:17.531315 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:17.533331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:17.569577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dafbd-f767-43ba-9052-a22301828e66 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:17.653089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:17.654007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:17.654302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:17.654622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] PATH_INFO: `/v3/projects/834d98cfb7a2427baaef31654abd9774` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:17.656014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] RBAC: Authorizing `identity:delete_project(project_id=834d98cfb7a2427baaef31654abd9774)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:17.658480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:17.784772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834d98cfb7a2427baaef31654abd9774', 'request_id': 'req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.791650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834d98cfb7a2427baaef31654abd9774', 'request_id': 'req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.842601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:17.870838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834d98cfb7a2427baaef31654abd9774', 'request_id': 'req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.882513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] The token cache is being invalidate because project 834d98cfb7a2427baaef31654abd9774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:17.882804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.883487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c62fb6d3-6b2a-4ace-8c4e-d8dcd936848c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:17.885120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 980/1942] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:17 2022] DELETE /identity/v3/projects/834d98cfb7a2427baaef31654abd9774 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:17.889961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:17.890256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:17.890540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:17.905516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:17.909075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:17.910402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:17.971141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 963/1943] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:17.979888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:17.980182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:17.980514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:17.999257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:18.001650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:18.011065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 981/1944] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:18.015347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.027227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c1713-8cd1-4074-9a46-3b2d218e25a0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.030778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.031458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.031750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.032035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.032860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:18.033153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.034780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.051053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 964/1945] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:18.053273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.063628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917d3e4-c922-4f5b-8e3b-27a00d71f883 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.068722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.069518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.069816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.070103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.070820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.072777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.147536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 982/1946] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:18 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:18.151824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.161874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312587b1-0db2-47ef-9b29-3d61b6d71923 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.228780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.229765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.230050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.230330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.231040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.233046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.306072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 965/1947] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:18 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:18.310296 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.320932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea2e89-ea8f-4348-8922-1cb7d8b8d987 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.325527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.326200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.326488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.326773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.327513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.329162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.340044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 983/1948] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:18.344413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.355312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fb9e5-1b70-46cd-a6e3-8d19e4c5c245 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.359899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.360631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.361089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.361370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.362104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.363706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e75852-4325-46bd-8a24-5a961446e1c0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.374441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 966/1949] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:18.378691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.389502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2dc08-c76b-4eec-a1e2-0d1902db1d58 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.394220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.395142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.395529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.395817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] PATH_INFO: `/v3/projects/568708c28d8c4df78494d70ea4e066ed/users/091ae25409914c8ab96977f0ca5b257d/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.409493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.415082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568708c28d8c4df78494d70ea4e066ed, user_id=091ae25409914c8ab96977f0ca5b257d, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.416144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.495756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 984/1950] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:18 2022] PUT /identity/v3/projects/568708c28d8c4df78494d70ea4e066ed/users/091ae25409914c8ab96977f0ca5b257d/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:18.506501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.509974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.509974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.535239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] MFA Rules not processed for user `091ae25409914c8ab96977f0ca5b257d`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:18.537234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:18.571333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.626092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 967/1951] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:18.627332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.672019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.695101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3509f-d74e-40d2-b063-3e37a7cda813 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.700628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.701297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.701590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.701877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.702637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.704614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.711248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.793530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 985/1952] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:18 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:18.798152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.810559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005b635-2819-437b-8be5-d50f6d3c32b2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.873542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.874238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.874534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.874857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.875557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.877542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.899744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.958133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 968/1953] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:18 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:18.962028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:18.972273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00533e-751a-475b-a28b-d10ffad5279c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:18.977552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:18.978274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:18.978804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:18.979530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:18.981380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:18.981840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:18.983536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:18.994531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 986/1954] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:18.998676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:19.009758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54fa7d-31ab-4212-8f86-e3797830780d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:19.015042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:19.016578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.016865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.017146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] PATH_INFO: `/v3/projects/93335891f5fc45c6bff3c169b09a2da3/users/9c917b5601804c68995c1d813fa0b7fd/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.032751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93335891f5fc45c6bff3c169b09a2da3, user_id=9c917b5601804c68995c1d813fa0b7fd, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:19.035500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:19.111350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 969/1955] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:18 2022] PUT /identity/v3/projects/93335891f5fc45c6bff3c169b09a2da3/users/9c917b5601804c68995c1d813fa0b7fd/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:19.115434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:19.126529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87334cd6-0bbc-46f7-9b2c-65a501aa0e59 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:19.168573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.184777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:19.185476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.185771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.186089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.186869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:19.188600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58f4b5-41a1-4e08-9207-e31d5e93c672 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:19.191626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.204658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 987/1956] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:19.205847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:19.218351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06962b3b-6896-4ee1-9f78-ad12f2fea3b7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:19.223292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:19.224338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.224631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.224916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] PATH_INFO: `/v3/projects/93335891f5fc45c6bff3c169b09a2da3/users/9c917b5601804c68995c1d813fa0b7fd/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.226919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93335891f5fc45c6bff3c169b09a2da3, user_id=9c917b5601804c68995c1d813fa0b7fd, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:19.230130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:19.241218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 970/1957] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:19 2022] PUT /identity/v3/projects/93335891f5fc45c6bff3c169b09a2da3/users/9c917b5601804c68995c1d813fa0b7fd/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:19.252500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.252804 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.253087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.259793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.281113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] MFA Rules not processed for user `9c917b5601804c68995c1d813fa0b7fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:19.283214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:19.328869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.352276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 988/1958] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:19.364247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:19.365985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:19.421901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1179810-c4a4-4daf-aa36-7fb247afaa70 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:19.426027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.428397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:19.429063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.432514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.432514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.439164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:19.445648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:19.455952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 971/1959] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:19.482625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.581348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:19.590524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.591225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4a690-bc54-4321-bfac-7a434744016a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:19.594891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:19.599515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:19.599515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:19.599515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:19.649465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.657191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:19.658180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32ff6-6671-407c-ad94-ebbf87dcdd68 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:19.668055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 989/1960] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:19.677554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.853887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:19.944908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.088046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.156425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:20.197179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.220427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e51fed-6edf-4d64-95d4-fe0753db2a7a service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:20.225951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446212-f753-421b-8923-67001912e1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:20.226835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446212-f753-421b-8923-67001912e1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:20.227130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446212-f753-421b-8923-67001912e1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:20.227421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446212-f753-421b-8923-67001912e1db admin admin] PATH_INFO: `/v3/users/091ae25409914c8ab96977f0ca5b257d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:20.228669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9446212-f753-421b-8923-67001912e1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=091ae25409914c8ab96977f0ca5b257d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:20.230884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9446212-f753-421b-8923-67001912e1db admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:20.328813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.443449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.452500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091ae25409914c8ab96977f0ca5b257d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:20.453178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091ae25409914c8ab96977f0ca5b257d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:20.466132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091ae25409914c8ab96977f0ca5b257d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:20.472920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091ae25409914c8ab96977f0ca5b257d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:20.539548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091ae25409914c8ab96977f0ca5b257d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:20.546053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 972/1961] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:20 2022] DELETE /identity/v3/users/091ae25409914c8ab96977f0ca5b257d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:20.551087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.551605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:20.551970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:20.552390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:20.570580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:20.571780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:20.630864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 990/1962] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:20.640548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:20.642997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:20.670979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.703913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446212-f753-421b-8923-67001912e1db admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:20.708825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:20.709481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:20.709771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:20.710052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:20.719358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:20.720883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:20.731045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 973/1963] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:20.773515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:20.854539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:21.187217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:21.188911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:21.203880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a17f3-aa69-4b55-a782-d683c2d6b3d1 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:21.209064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:21.210108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:21.210555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:21.210833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] PATH_INFO: `/v3/projects/568708c28d8c4df78494d70ea4e066ed` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:21.212079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] RBAC: Authorizing `identity:delete_project(project_id=568708c28d8c4df78494d70ea4e066ed)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:21.214600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:21.218930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:21.305616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:21.380812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568708c28d8c4df78494d70ea4e066ed', 'request_id': 'req-5482c030-a25d-4fac-9040-9a8b0a97595e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.386999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568708c28d8c4df78494d70ea4e066ed', 'request_id': 'req-5482c030-a25d-4fac-9040-9a8b0a97595e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.493640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568708c28d8c4df78494d70ea4e066ed', 'request_id': 'req-5482c030-a25d-4fac-9040-9a8b0a97595e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.505546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] The token cache is being invalidate because project 568708c28d8c4df78494d70ea4e066ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:21.505928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.506630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.508356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 991/1964] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:21 2022] DELETE /identity/v3/projects/568708c28d8c4df78494d70ea4e066ed => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:21.512463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:21.522623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee91d2-473e-40ed-8653-6ae12e20b5bc service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:21.582836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:21.583517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:21.583849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:21.584132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] PATH_INFO: `/v3/users/9c917b5601804c68995c1d813fa0b7fd` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:21.585385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c917b5601804c68995c1d813fa0b7fd)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:21.587337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:21.608415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:21.758881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c917b5601804c68995c1d813fa0b7fd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.759626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c917b5601804c68995c1d813fa0b7fd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.772796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c917b5601804c68995c1d813fa0b7fd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.779553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c917b5601804c68995c1d813fa0b7fd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.848007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-09d6c7ec-1b00-4619-91d9-4258adc15074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c917b5601804c68995c1d813fa0b7fd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:21.855744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 974/1965] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:21 2022] DELETE /identity/v3/users/9c917b5601804c68995c1d813fa0b7fd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:21.860166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:21.951516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:22.214029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:22.359057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:22.360766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:22.374621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c030-a25d-4fac-9040-9a8b0a97595e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:22.444453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:22.445346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:22.445644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:22.445914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] PATH_INFO: `/v3/projects/93335891f5fc45c6bff3c169b09a2da3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:22.447215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93335891f5fc45c6bff3c169b09a2da3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:22.449421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:22.576359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93335891f5fc45c6bff3c169b09a2da3', 'request_id': 'req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:22.583179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93335891f5fc45c6bff3c169b09a2da3', 'request_id': 'req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:22.699558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93335891f5fc45c6bff3c169b09a2da3', 'request_id': 'req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:22.716604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] The token cache is being invalidate because project 93335891f5fc45c6bff3c169b09a2da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:22.716905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:22.717657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9ed96080-32e8-4245-b6d5-d0a3e7a77c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:22.719621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 992/1966] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:22 2022] DELETE /identity/v3/projects/93335891f5fc45c6bff3c169b09a2da3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:22.725451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5936-663b-487b-8012-b34c7059f455 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:22.728041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5936-663b-487b-8012-b34c7059f455 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:22.728442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5936-663b-487b-8012-b34c7059f455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:22.750256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-998f5936-663b-487b-8012-b34c7059f455 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:22.751629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-998f5936-663b-487b-8012-b34c7059f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:22.809929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:22.820030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 975/1967] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:22.833728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:22.833728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:22.833728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:22.850229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:22.851773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:22.860619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 993/1968] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:22.864968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5936-663b-487b-8012-b34c7059f455 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:22.875447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5936-663b-487b-8012-b34c7059f455 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:22.881234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:22.881924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:22.882217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:22.882499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:22.883253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:22.883566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:22.885380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:22.902723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 976/1969] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:26:22.904432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:22.915052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a70831-0e20-4b03-9483-5eb56d6723f0 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:22.920889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:22.921332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:22.921690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:22.921975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:22.922802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:22.924816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.050258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 994/1970] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:22 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:23.055107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:23.066482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a9f8-754c-4118-9f93-9eaba0b5d2b8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:23.130759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:23.131392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.131704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.131985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.132747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:23.134686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.231838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 977/1971] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:23 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:23.236293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:23.248372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9068e-f385-4ef9-a945-ed34c863e1a9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:23.253183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:23.253884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.254281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.255578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.256061 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:23.257766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.268901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 995/1972] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:23.273207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:23.283410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc365699-2166-4bf8-a8c9-22aa6ead5665 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:23.288061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:23.288787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.289076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.289359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.290097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:23.291713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ce1f-630d-44ea-b47a-be2e914c2c60 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.302505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 978/1973] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:23.307132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:23.317983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd7a50-8cae-4314-ae80-d58f45c02ba6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:23.322714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:23.323706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.323993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.324276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] PATH_INFO: `/v3/projects/bd433cf4e58b4bbe9547853933c51744/users/4d02acf2a82848e8a52c489b8b927993/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.341148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd433cf4e58b4bbe9547853933c51744, user_id=4d02acf2a82848e8a52c489b8b927993, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:23.344073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.439357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 996/1974] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:23 2022] PUT /identity/v3/projects/bd433cf4e58b4bbe9547853933c51744/users/4d02acf2a82848e8a52c489b8b927993/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:23.457016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.457016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.457016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.486627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] MFA Rules not processed for user `4d02acf2a82848e8a52c489b8b927993`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:23.487884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:23.500698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:23.572199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 979/1975] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:23.584912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:23.586612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:23.647045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2651e-e637-43da-ba7c-67ac86a7ed76 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:23.650756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:23.651414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:23.651707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:23.651996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:23.660549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:23.662553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816842e3-390c-476a-a7a3-3f8ba0638f5b service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:23.671965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 997/1976] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:23.917303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:24.373651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:24.562418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:24.705866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:25.269652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:25.271810 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:25.341412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa830-8d45-46a8-9653-ba3843190d93 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:25.346377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:25.347280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:25.347577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:25.348062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] PATH_INFO: `/v3/users/815b12a77d4d4649804d5327ddeaca62` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:25.350172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815b12a77d4d4649804d5327ddeaca62)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:25.352245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:25.611573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815b12a77d4d4649804d5327ddeaca62'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:25.612192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815b12a77d4d4649804d5327ddeaca62'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:25.625045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815b12a77d4d4649804d5327ddeaca62'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:25.632983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815b12a77d4d4649804d5327ddeaca62'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:25.757261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815b12a77d4d4649804d5327ddeaca62'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:25.764264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 980/1977] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:25 2022] DELETE /identity/v3/users/815b12a77d4d4649804d5327ddeaca62 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:25.772583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:25.773125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:25.773245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:25.795140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:25.797434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:25.853424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:25.855209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:25.883474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 998/1978] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:25.894416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:25.897376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:25.919404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:25.937156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5c450-6cd8-4045-9ad1-cd202bcc6ae5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:25.940586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:25.944095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:25.945242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:25.947534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:25.947534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] PATH_INFO: `/v3/users/8579d20c2ebc4848a71096e14f5f1455` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:25.947534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8579d20c2ebc4848a71096e14f5f1455)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:25.949632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:25.982600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d2de8-df0f-46fc-b8fa-65eea9fdb6fd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:25.985058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:25.985722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:25.986013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:25.986294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:25.996632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:25.998089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7fdab-9e5e-4b68-a074-98273e0bcbd2 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:26.008019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 999/1979] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:26.165274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8579d20c2ebc4848a71096e14f5f1455'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:26.165955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8579d20c2ebc4848a71096e14f5f1455'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:26.179991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8579d20c2ebc4848a71096e14f5f1455'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:26.187353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8579d20c2ebc4848a71096e14f5f1455'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:26.328197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8579d20c2ebc4848a71096e14f5f1455'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:26.336260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 981/1980] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:25 2022] DELETE /identity/v3/users/8579d20c2ebc4848a71096e14f5f1455 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:26.341615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:26.342088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:26.342389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:26.363196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:26.366016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:26.436783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1000/1981] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:26.447948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:26.452124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:26.515385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ead78-1201-45bd-a72d-cac0802db846 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:26.518991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:26.519647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:26.520050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:26.521142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:26.532092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:26.533606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:26.544493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 982/1982] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:26.760184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:26.762086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:26.775252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cb0a4-bc8a-4ce3-af3a-c4f83750749c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:26.779992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:26.780924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:26.781217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:26.781679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] PATH_INFO: `/v3/users/4d02acf2a82848e8a52c489b8b927993` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:26.782983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d02acf2a82848e8a52c489b8b927993)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:26.785034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:26.822724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:26.825180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:26.839252 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260336b-5883-4928-a72c-1e275ee6cc14 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:26.859748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:26.862328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:26.864618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:26.865210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] PATH_INFO: `/v3/projects/c780c3e3d141486cbcd8250dae457e32` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:26.867310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c780c3e3d141486cbcd8250dae457e32)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:26.870745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:27.066885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c780c3e3d141486cbcd8250dae457e32', 'request_id': 'req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.074283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d02acf2a82848e8a52c489b8b927993'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.077249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c780c3e3d141486cbcd8250dae457e32', 'request_id': 'req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.085868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d02acf2a82848e8a52c489b8b927993'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.102387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d02acf2a82848e8a52c489b8b927993'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.103202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d02acf2a82848e8a52c489b8b927993'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.103991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d02acf2a82848e8a52c489b8b927993'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.173482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c780c3e3d141486cbcd8250dae457e32', 'request_id': 'req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.178468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:27.187372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] The token cache is being invalidate because project c780c3e3d141486cbcd8250dae457e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:27.187748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.188376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.190040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 983/1983] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:26 2022] DELETE /identity/v3/projects/c780c3e3d141486cbcd8250dae457e32 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:27.193111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:27.194430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.290995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ba3f-6d42-4150-af5d-62bd0e1d3f89 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:27.296482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:27.297612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:27.297981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:27.298338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] PATH_INFO: `/v3/projects/2ab4321ed4b24d2aa1fdcc565426b682` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:27.299820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab4321ed4b24d2aa1fdcc565426b682)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:27.302065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:27.344741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1001/1984] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:26 2022] DELETE /identity/v3/users/4d02acf2a82848e8a52c489b8b927993 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:27.346195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:27.347803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.409177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:27.448028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab4321ed4b24d2aa1fdcc565426b682', 'request_id': 'req-0974a236-2929-4ee4-99fc-c65472f835c4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.456075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a82be-42ab-4cfe-9b8b-abb2f5ea03d4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:27.459093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab4321ed4b24d2aa1fdcc565426b682', 'request_id': 'req-0974a236-2929-4ee4-99fc-c65472f835c4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.536900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:27.536900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:27.536900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:27.536900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] PATH_INFO: `/v3/users/26417e93b1cf4d48ad0d67b140230a06` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:27.537499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26417e93b1cf4d48ad0d67b140230a06)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:27.537499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:27.557250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab4321ed4b24d2aa1fdcc565426b682', 'request_id': 'req-0974a236-2929-4ee4-99fc-c65472f835c4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.573283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] The token cache is being invalidate because project 2ab4321ed4b24d2aa1fdcc565426b682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:27.573763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.574447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0974a236-2929-4ee4-99fc-c65472f835c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.576221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 984/1985] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:27 2022] DELETE /identity/v3/projects/2ab4321ed4b24d2aa1fdcc565426b682 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:27.577628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:27.579548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:27.579855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:27.607693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:27.611340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.617837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 985/1986] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:27.618500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:27.620137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.635331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea57b20-bb2e-4d78-9728-119377192767 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:27.640125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:27.642122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:27.642122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:27.642122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] PATH_INFO: `/v3/users/3e5ed8131bfc4312a193922d0c4c4d49` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:27.642828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5ed8131bfc4312a193922d0c4c4d49)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:27.644836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:27.720827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26417e93b1cf4d48ad0d67b140230a06'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.730288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26417e93b1cf4d48ad0d67b140230a06'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.743966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26417e93b1cf4d48ad0d67b140230a06'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.744775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26417e93b1cf4d48ad0d67b140230a06'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.745403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26417e93b1cf4d48ad0d67b140230a06'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.847551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1002/1987] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:27 2022] DELETE /identity/v3/users/26417e93b1cf4d48ad0d67b140230a06 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:27.848873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:27.850346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.871091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5ed8131bfc4312a193922d0c4c4d49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.872043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5ed8131bfc4312a193922d0c4c4d49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.889300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5ed8131bfc4312a193922d0c4c4d49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.895236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=17,keystone:SELECT=82,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:27.897683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5ed8131bfc4312a193922d0c4c4d49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.926857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831229c-5c71-40a6-a5e2-77cbd714904c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:27.931584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:27.932466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:27.932763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:27.933100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:27.934974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:27.951832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:27.956692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5ed8131bfc4312a193922d0c4c4d49'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:27.963339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 986/1988] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:27 2022] DELETE /identity/v3/users/3e5ed8131bfc4312a193922d0c4c4d49 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:28.008125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:28.009743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:28.022036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1003/1989] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:28.401649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:28.403096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:28.488353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d6df1-0d32-4288-8406-f3a311fb5fce admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:28.493342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:28.494223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:28.494514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:28.498954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] PATH_INFO: `/v3/projects/38fc74ec2547460aaf7056068bd30668` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:28.498954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38fc74ec2547460aaf7056068bd30668)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:28.498954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:28.526522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:28.528369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:28.542162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb2ec-c5ff-4403-b200-ac34b1bb6cf2 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:28.547803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:28.548757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:28.549017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:28.549293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] PATH_INFO: `/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:28.550633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eef6427bbe64c5fbfdd21ead3fd8951)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:28.558007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:28.666733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38fc74ec2547460aaf7056068bd30668', 'request_id': 'req-47909696-5bb7-4f6f-a937-773f22388958'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.674554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38fc74ec2547460aaf7056068bd30668', 'request_id': 'req-47909696-5bb7-4f6f-a937-773f22388958'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.679122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eef6427bbe64c5fbfdd21ead3fd8951', 'request_id': 'req-ee434292-07ba-4782-9b6c-2416fc3baf0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.686363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eef6427bbe64c5fbfdd21ead3fd8951', 'request_id': 'req-ee434292-07ba-4782-9b6c-2416fc3baf0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.807855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38fc74ec2547460aaf7056068bd30668', 'request_id': 'req-47909696-5bb7-4f6f-a937-773f22388958'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.819372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] The token cache is being invalidate because project 38fc74ec2547460aaf7056068bd30668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:28.819666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.820364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.822138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 987/1990] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:28 2022] DELETE /identity/v3/projects/38fc74ec2547460aaf7056068bd30668 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:28.827152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:28.827152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:28.895955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47909696-5bb7-4f6f-a937-773f22388958 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:28.922460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:28.956161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:28.957068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:28.957446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:28.958212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] PATH_INFO: `/v3/projects/bd433cf4e58b4bbe9547853933c51744` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:28.959688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd433cf4e58b4bbe9547853933c51744)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:28.961346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eef6427bbe64c5fbfdd21ead3fd8951', 'request_id': 'req-ee434292-07ba-4782-9b6c-2416fc3baf0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.961827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:28.974029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] The token cache is being invalidate because project 0eef6427bbe64c5fbfdd21ead3fd8951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:28.974481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.975249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ee434292-07ba-4782-9b6c-2416fc3baf0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:28.977046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1004/1991] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:28 2022] DELETE /identity/v3/projects/0eef6427bbe64c5fbfdd21ead3fd8951 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:28.978475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564826c1-d94d-4db5-a3b9-51f75a911600 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:28.978772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564826c1-d94d-4db5-a3b9-51f75a911600 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:28.979052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564826c1-d94d-4db5-a3b9-51f75a911600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:28.999441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-564826c1-d94d-4db5-a3b9-51f75a911600 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.000718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-564826c1-d94d-4db5-a3b9-51f75a911600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.009637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1005/1992] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.010929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80d6-5b8e-4a88-a389-f440b14a4c0f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.011214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80d6-5b8e-4a88-a389-f440b14a4c0f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.011522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a80d6-5b8e-4a88-a389-f440b14a4c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.030320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-693a80d6-5b8e-4a88-a389-f440b14a4c0f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.031556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-693a80d6-5b8e-4a88-a389-f440b14a4c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.039525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1006/1993] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.040835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36eac-434a-404d-bcee-bceab51fb03b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.041122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36eac-434a-404d-bcee-bceab51fb03b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.041990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca36eac-434a-404d-bcee-bceab51fb03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.063068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-aca36eac-434a-404d-bcee-bceab51fb03b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.064399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-aca36eac-434a-404d-bcee-bceab51fb03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.081366 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd433cf4e58b4bbe9547853933c51744', 'request_id': 'req-ab99211d-4317-44d4-95bc-b663d0cf8b5b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:29.088922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd433cf4e58b4bbe9547853933c51744', 'request_id': 'req-ab99211d-4317-44d4-95bc-b663d0cf8b5b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:29.132983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1007/1994] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.134681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.135058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.135537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.155200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.156241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.156621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd433cf4e58b4bbe9547853933c51744', 'request_id': 'req-ab99211d-4317-44d4-95bc-b663d0cf8b5b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:29.165461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1008/1995] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.166489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.169061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] The token cache is being invalidate because project bd433cf4e58b4bbe9547853933c51744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:29.169527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:29.170203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:29.171819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 988/1996] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:28 2022] DELETE /identity/v3/projects/bd433cf4e58b4bbe9547853933c51744 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:29.175111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.175111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.181049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7049aa-2852-42f6-a80a-64b675264770 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.187031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.187711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.187999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.188285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.188829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99211d-4317-44d4-95bc-b663d0cf8b5b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.189082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:29.189390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.191334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762a51f-2473-4784-8dac-cfdc0870790e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.195247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.195927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.196224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.196545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.197302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:29.197626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.199659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.204830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1009/1997] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:29.206802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.206802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.206802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.212880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 989/1998] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:29.213720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.215393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.217277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:29.229401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e42f8-f7f9-4df1-b9c7-1b0a6a06cd5b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.231625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.232877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.234852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.235537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.235884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.236096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.236838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.239501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fece79-9feb-44e1-926d-59e2f20b1574 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.241803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1010/1999] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.242678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.252767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4e34d-6886-4eff-a43c-33b0b7c50a76 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.258459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.259133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.259428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.259782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.266202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.266202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.353677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 990/2000] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:29.353976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1011/2001] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:29.355279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.355577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.355868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.358764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.375920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:29.378322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:29.435979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827734-9d87-4790-a210-12f32fae4667 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.456021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 991/2002] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:29.456958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.475703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f997aa7-8977-4e86-92b5-38e550c5cdff None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.481830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.482553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.482840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.483329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.484072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.486087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.513121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.513121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.513121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.513121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.513504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.513504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.564172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 992/2003] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:29.565776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.575669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:29.579931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a4630-f82f-4776-8800-c6b1ef133dbc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.585722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1012/2004] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/users => generated 282 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:29.585722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.588385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.589270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.589565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.590241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.590996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:29.591472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.593131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.596872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b611-6dcb-448e-a927-7d32dea141b7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.603361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.604068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.604648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.604933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.605928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.607169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 993/2005] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:29.607695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.608350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.624560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1013/2006] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:29.625538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.629166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b55f1-b7fd-4ea6-9291-79a2c790a7fb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.633148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.634126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.634752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.635094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.635923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.636751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206e419-ed4a-4e83-8035-0c9ad8321ae4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.637615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.643944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.644734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.645445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.645801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.646554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.651081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 994/2007] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:29.651223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.651303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.659383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500e65a-b9d9-4aff-b46e-e9c088774886 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.667150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.667949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.668250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.668608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.669349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.671006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.682304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 995/2008] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:29.683295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.694477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74fb0-1cc0-4a9a-904e-f0c526681f27 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.699264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.699929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.700219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.700534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.701268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.702886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.713919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 996/2009] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:29.714713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.724267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b738e9-f42b-46bd-a65a-c74f7af08c6d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.728929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.730063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.730352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.730635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] PATH_INFO: `/v3/projects/87a2c6be97a44fdd99e20e50a791da2b/users/515504022e2c4e6faa46f6b68c58c438/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.746734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a2c6be97a44fdd99e20e50a791da2b, user_id=515504022e2c4e6faa46f6b68c58c438, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.749455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.788079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1014/2010] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:29.789025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.805003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 997/2011] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:29 2022] PUT /identity/v3/projects/87a2c6be97a44fdd99e20e50a791da2b/users/515504022e2c4e6faa46f6b68c58c438/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:29.805972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:29.855159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:29.891261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f619baa-409e-4274-b92a-6bff5ecdd483 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.922602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527c3b-de5f-44d3-8875-688c6d518b6a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:29.924799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.925493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.925783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.926762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.927534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.929802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5932a-7fac-4fbf-b82b-9c5e815b7ae2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:29.968593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:29.969496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:29.969795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:29.970080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] PATH_INFO: `/v3/projects/41512c19609541879690cbc5cfc963e1/users/17b7e7c217174e318b0ab669a94eacf7/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:29.986197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41512c19609541879690cbc5cfc963e1, user_id=17b7e7c217174e318b0ab669a94eacf7, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:29.989609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.033944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 998/2012] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:29 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:30.033944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c8c4d-d80a-4d9a-9d7e-100cfb92cdc7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.033944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c8c4d-d80a-4d9a-9d7e-100cfb92cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.033944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c8c4d-d80a-4d9a-9d7e-100cfb92cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.054820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1015/2013] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:29 2022] PUT /identity/v3/projects/41512c19609541879690cbc5cfc963e1/users/17b7e7c217174e318b0ab669a94eacf7/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:30.059039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.061090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0c4c8c4d-d80a-4d9a-9d7e-100cfb92cdc7 None None] MFA Rules not processed for user `515504022e2c4e6faa46f6b68c58c438`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:30.062402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0c4c8c4d-d80a-4d9a-9d7e-100cfb92cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:30.123094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54592044-023a-4e3c-ba26-69cf7279dff1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.128140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.128841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.129133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.130109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.131574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.132385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 999/2014] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:30.133309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.133877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.134183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.134466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.147179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1016/2015] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:30.147179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.149006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:30.160030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:30.163501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] MFA Rules not processed for user `17b7e7c217174e318b0ab669a94eacf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:30.164985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:30.225438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:30.234182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726dd18-1a52-4c24-aa1b-7124fedb3a07 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.236977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1000/2016] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:30.237950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.239250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.240806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.241136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.241510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.250511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4654f6c-eadf-472c-b8b5-1b34db3e75c4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.250715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.252108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.256821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.257499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.257787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.258066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.258844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.260695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.262452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1017/2017] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:30.263270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.272341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1001/2018] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:30.276576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.288280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bf5a5-5266-4cc4-802e-05b65af003a4 service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.288280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.288280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.288280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.288848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.288848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.289200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.291986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3b54e-57e5-43c9-be3e-3ac76c8af359 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.299881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.300854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.301806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.302163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] PATH_INFO: `/v3/projects/2d88280f17e84da79842ac2e0780fb97/users/df40fa149a5e478b9e65c118d1977488/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.321821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d88280f17e84da79842ac2e0780fb97, user_id=df40fa149a5e478b9e65c118d1977488, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.325330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace2b77-ab1e-4ae2-9009-d2b38062d3d8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.420676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1018/2019] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:30 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:30.424701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.444519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1002/2020] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:30 2022] PUT /identity/v3/projects/2d88280f17e84da79842ac2e0780fb97/users/df40fa149a5e478b9e65c118d1977488/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:30.454639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.454639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.454922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.493504 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] MFA Rules not processed for user `df40fa149a5e478b9e65c118d1977488`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:30.494830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:30.522622 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d93d4-2b20-4066-a3f5-216960e0bc87 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.583234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1003/2021] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:30.590737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.592139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:30.603328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.604010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.604579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.604858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.605561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.607538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.654127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f2887-dfbe-4082-a075-c11a33c5d485 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.657849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.658503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.658817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.659097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.668063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.669750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.674744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:30.681887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1004/2022] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:30.700757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1019/2023] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:30 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:30.704833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.716184 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25d826-a54f-4dd9-bc33-cf9a3b6c50bc service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.720996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.721664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.721955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.722234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.723331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.724984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.736033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1005/2024] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:30.741222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.756953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05890-d590-4d03-9d42-20de0027cf17 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.762357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.763265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.763562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.763844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] PATH_INFO: `/v3/projects/15b8701433d2402682d6b02d6313040c/users/87ecf9d8bc454fc28beb9510a0e553bb/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.780561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b8701433d2402682d6b02d6313040c, user_id=87ecf9d8bc454fc28beb9510a0e553bb, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.783446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.872289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1020/2025] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:30 2022] PUT /identity/v3/projects/15b8701433d2402682d6b02d6313040c/users/87ecf9d8bc454fc28beb9510a0e553bb/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:30.876666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.887499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4934db-13cf-4da1-9bb5-dcd298f41e7a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.950788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.951499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.951744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.952203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.952943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.954621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd53ce6-a991-4af7-b0d1-4e218fe9f37e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:30.965811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1006/2026] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:30.970123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:30.981517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8c7e-e1d5-47a4-9b57-65c948508f55 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:30.986419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:30.987231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:30.987703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:30.987959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] PATH_INFO: `/v3/projects/15b8701433d2402682d6b02d6313040c/users/87ecf9d8bc454fc28beb9510a0e553bb/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:30.990686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b8701433d2402682d6b02d6313040c, user_id=87ecf9d8bc454fc28beb9510a0e553bb, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:30.993410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:31.005563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1021/2027] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:30 2022] PUT /identity/v3/projects/15b8701433d2402682d6b02d6313040c/users/87ecf9d8bc454fc28beb9510a0e553bb/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:31.019650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:31.019965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:31.020238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:31.050794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] MFA Rules not processed for user `87ecf9d8bc454fc28beb9510a0e553bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:31.052670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:31.155308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1007/2028] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:31.261855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:31.263701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:31.311120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:31.357070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a1baf-f3dc-4c2f-a876-b7ce158002f2 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:31.364881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:31.365613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:31.366086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:31.366915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:31.378314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:31.381431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60694b9-0615-4a06-b117-7802d7600f80 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:31.395337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1022/2029] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:31.654522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:31.664734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4654b9d-969c-4520-84d6-6ed09ccbbf46 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:31.669265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:31.669948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:31.670240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:31.670532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:31.743956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:31.745813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4ad32-cfea-41f4-8163-bfba9160c4bf service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:31.761479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1008/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:37.384781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:37.384781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:37.384781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:37.386026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1023/2031] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:26:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:26:37.390338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:37.390883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:37.391032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:37.413273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:37.414789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:37.420586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:37.508408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1009/2032] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:26:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:37.735036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:37.739424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:37.747104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:37.749338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:37.766501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089933d1-c7f7-459d-8147-00b30a1d0020 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:37.778420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:37.779368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:37.779847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:37.780256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:37.790996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:37.796519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:37.808743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1010/2033] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:37.830930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64c33c-32dc-4170-9a81-f275ab81fc14 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:37.839951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:37.839951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:37.839951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:37.839951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] PATH_INFO: `/v3/users/e9f4e1de59734b3f87a982a50b5b2f91` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:37.840903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f4e1de59734b3f87a982a50b5b2f91)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:37.841176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:38.084441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f4e1de59734b3f87a982a50b5b2f91'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.092127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f4e1de59734b3f87a982a50b5b2f91'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.108982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f4e1de59734b3f87a982a50b5b2f91'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.109748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f4e1de59734b3f87a982a50b5b2f91'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.110367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f6cedc86-80a1-4604-aab4-58aa1f07049f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f4e1de59734b3f87a982a50b5b2f91'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.236701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1024/2034] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:37 2022] DELETE /identity/v3/users/e9f4e1de59734b3f87a982a50b5b2f91 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:38.502275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:38.523478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2c481-1bdf-4e53-b09d-87e1293bab28 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:38.618744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:38.619939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:38.620237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:38.620603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] PATH_INFO: `/v3/projects/5ce90d7b4e2540d8a3417db6900d9243` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:38.622348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ce90d7b4e2540d8a3417db6900d9243)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:38.624623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:38.773437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ce90d7b4e2540d8a3417db6900d9243', 'request_id': 'req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.780500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ce90d7b4e2540d8a3417db6900d9243', 'request_id': 'req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.853859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ce90d7b4e2540d8a3417db6900d9243', 'request_id': 'req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.866895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] The token cache is being invalidate because project 5ce90d7b4e2540d8a3417db6900d9243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:38.867216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.867899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e32ed67f-df40-4467-8ce0-dfd2abcc9cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:38.869691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1011/2035] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:38 2022] DELETE /identity/v3/projects/5ce90d7b4e2540d8a3417db6900d9243 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:38.874902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:38.875472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:38.875472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:38.896202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:38.897747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:38.975720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:38.999791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1025/2036] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:39.018007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.018317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.018605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.040398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:39.040978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:39.051391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1012/2037] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:39.056142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.070220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b9ca04-f545-467f-a46d-a93777e999ff None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.075674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.076386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.076685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.077016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.077745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:39.078065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.081118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.100197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1026/2038] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:39.106930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.123793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e2396-7c06-4cdb-b51d-e6110ef05b2d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.130743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.131435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.131728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.132020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.132773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.134763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.151096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:39.259222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1013/2039] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/projects => generated 403 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:39.264096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.275346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfca927-9b80-4686-bf81-1d6ddd5f2306 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.345745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.346456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.348220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.348220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.348220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.350423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.464716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1027/2040] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/users => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:39.469114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.482148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae29d77-e9e0-4733-aeeb-af8a2e220975 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.487490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.488209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.488640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.489010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.489811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.491565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.512209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1014/2041] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:39.518081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.529941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf0020-d592-4371-98d4-e82e0a8b3166 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.535054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.536412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.537337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.537337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.537881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.539620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98d0d1-9ddc-4344-b28f-6b114d289a83 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.552574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1028/2042] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:39.558029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.577332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ccde3-9edc-42b8-9423-3a2de4553c62 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.584864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.585843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.586243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.586994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] PATH_INFO: `/v3/projects/5588ee49def8481d9b1b6d413371958b/users/b6a63d47d25c45c7bf7319f3639bc6a9/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.607781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5588ee49def8481d9b1b6d413371958b, user_id=b6a63d47d25c45c7bf7319f3639bc6a9, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.610801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.700026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1015/2043] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:39 2022] PUT /identity/v3/projects/5588ee49def8481d9b1b6d413371958b/users/b6a63d47d25c45c7bf7319f3639bc6a9/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:39.710564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.710935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.711289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.739380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] MFA Rules not processed for user `b6a63d47d25c45c7bf7319f3639bc6a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:39.740886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:39.808460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1029/2044] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:39.813107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.878389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad92b5-c216-47b8-9d2e-5ceac2726d9b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:39.883983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:39.884832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:39.884985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:39.885239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:39.885957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:39.887894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:39.969637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1016/2045] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:39.981361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:39.994890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6cd8f-4869-405c-a1a4-9acde3fd5b4e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.038895 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:40.056371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.057063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.057353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.061757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.061757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.061757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.192573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1030/2046] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:39 2022] POST /identity/v3/users => generated 309 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:40.196696 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:40.207558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20f45-d7db-41a9-baf8-e11ba5ff48fd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.213437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.214166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.214542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.214948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.215780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.217673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.230522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1017/2047] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:40.234720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:40.245722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a405f-6d67-4d88-90e9-dde1d43ad2c1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.250338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.251221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.251509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.251799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] PATH_INFO: `/v3/projects/6ff02128a22a48d5ac909f24cd678dee/users/c2e01c38424847878603a0586d8b0440/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.266791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff02128a22a48d5ac909f24cd678dee, user_id=c2e01c38424847878603a0586d8b0440, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.269733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.391185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1031/2048] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:40 2022] PUT /identity/v3/projects/6ff02128a22a48d5ac909f24cd678dee/users/c2e01c38424847878603a0586d8b0440/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:40.396135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:40.406889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e35d8-beb6-463b-b49a-131347e4c683 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.476535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.477226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.477523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.477812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.478567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.480488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f54f7-e801-4253-8fad-f736025bf4b9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.496989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1018/2049] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:40.499752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:40.512591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5219dc-d211-469b-bb5d-900dc708e933 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.517751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.519698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.519698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.519698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] PATH_INFO: `/v3/projects/6ff02128a22a48d5ac909f24cd678dee/users/c2e01c38424847878603a0586d8b0440/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.521777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ff02128a22a48d5ac909f24cd678dee, user_id=c2e01c38424847878603a0586d8b0440, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.524605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.538387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1032/2050] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:40 2022] PUT /identity/v3/projects/6ff02128a22a48d5ac909f24cd678dee/users/c2e01c38424847878603a0586d8b0440/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:40.548742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.549288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.549288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.579649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] MFA Rules not processed for user `c2e01c38424847878603a0586d8b0440`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:40.580940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:40.599229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:40.655092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1019/2051] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:40 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:40.666443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:40.667901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:40.741592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67739b60-2330-4ccd-ae3b-a7a5d03ed9cb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:40.745308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:40.745968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:40.746259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:40.746545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:40.762919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:40.764457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8fb147-1079-4c6c-b863-485ddb2cdbbb service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:40.774967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1033/2052] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:40 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:41.002539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:41.065453 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad75440-c547-4f26-8cb2-15f3e051cfa0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:41.070369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:41.071253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:41.071544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:41.071824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] PATH_INFO: `/v3/users/b6a63d47d25c45c7bf7319f3639bc6a9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:41.073011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6a63d47d25c45c7bf7319f3639bc6a9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:41.075005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:41.286392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6a63d47d25c45c7bf7319f3639bc6a9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:41.287019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6a63d47d25c45c7bf7319f3639bc6a9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:41.299240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6a63d47d25c45c7bf7319f3639bc6a9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:41.306791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6a63d47d25c45c7bf7319f3639bc6a9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:41.379163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6a63d47d25c45c7bf7319f3639bc6a9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:41.386068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1020/2053] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:41 2022] DELETE /identity/v3/users/b6a63d47d25c45c7bf7319f3639bc6a9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:41.390967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:41.391596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:41.391755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:41.410867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:41.412018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:41.479471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1034/2054] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:41.489820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:41.491644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:41.530478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:41.558645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515614a-d45a-4c63-964a-6d83cc8ba2b8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:41.562703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:41.563364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:41.563786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:41.564093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:41.574584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:41.576047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:41.586628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1021/2055] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:42.073216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:42.080661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:42.105843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f4ba4-d264-4163-b7c2-c6fb520dd2d2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:42.111152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:42.112271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:42.112598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:42.112883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] PATH_INFO: `/v3/projects/5588ee49def8481d9b1b6d413371958b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:42.114886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5588ee49def8481d9b1b6d413371958b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:42.116814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:42.256346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:42.277942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5588ee49def8481d9b1b6d413371958b', 'request_id': 'req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.286288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5588ee49def8481d9b1b6d413371958b', 'request_id': 'req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.367635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5588ee49def8481d9b1b6d413371958b', 'request_id': 'req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.384066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] The token cache is being invalidate because project 5588ee49def8481d9b1b6d413371958b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:42.384883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.385932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.390743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1035/2056] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:42 2022] DELETE /identity/v3/projects/5588ee49def8481d9b1b6d413371958b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:42.392341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:42.404725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235065a7-0e77-4a03-8ef8-185d2115cbf8 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:42.479952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:42.480921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:42.481208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:42.481497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] PATH_INFO: `/v3/users/c2e01c38424847878603a0586d8b0440` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:42.482744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e01c38424847878603a0586d8b0440)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:42.484812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:42.784353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e01c38424847878603a0586d8b0440'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.785185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e01c38424847878603a0586d8b0440'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.797490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e01c38424847878603a0586d8b0440'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.805911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e01c38424847878603a0586d8b0440'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.893049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d2335366-0709-4475-9d05-d85b16282792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e01c38424847878603a0586d8b0440'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:42.899847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1022/2057] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:42 2022] DELETE /identity/v3/users/c2e01c38424847878603a0586d8b0440 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:43.589882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:43.592735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:43.629763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1284c4-5353-4cba-bfaf-7e1e15c35aa8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:43.718276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:43.719252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:43.719525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:43.720005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] PATH_INFO: `/v3/projects/6ff02128a22a48d5ac909f24cd678dee` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:43.721277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ff02128a22a48d5ac909f24cd678dee)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:43.724665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:43.839280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ff02128a22a48d5ac909f24cd678dee', 'request_id': 'req-33e48c74-e38f-442e-9afb-dfc8bd93c19b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:43.848051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ff02128a22a48d5ac909f24cd678dee', 'request_id': 'req-33e48c74-e38f-442e-9afb-dfc8bd93c19b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:43.932088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ff02128a22a48d5ac909f24cd678dee', 'request_id': 'req-33e48c74-e38f-442e-9afb-dfc8bd93c19b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:43.944643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] The token cache is being invalidate because project 6ff02128a22a48d5ac909f24cd678dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:43.944971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:43.945758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-33e48c74-e38f-442e-9afb-dfc8bd93c19b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:43.947356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1036/2058] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:43 2022] DELETE /identity/v3/projects/6ff02128a22a48d5ac909f24cd678dee => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:43.952426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:43.952952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:43.953085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:43.975762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:43.976998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:44.038239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:44.055928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1023/2059] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:44.065446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca103-4f12-41f7-b192-120409464551 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.065842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca103-4f12-41f7-b192-120409464551 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.066774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca103-4f12-41f7-b192-120409464551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.094998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-67bca103-4f12-41f7-b192-120409464551 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:44.098250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-67bca103-4f12-41f7-b192-120409464551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:44.104747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1037/2060] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:44.109641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.120497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50be4a7-c471-48d3-a03f-066d425ede69 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.125952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.126655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.126943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.127225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.128176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:44.128534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.131003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.152298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1024/2061] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:44.152696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca103-4f12-41f7-b192-120409464551 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.163878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca103-4f12-41f7-b192-120409464551 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.169357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.173059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.173059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.173059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.173059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.173392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.312950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1038/2062] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:44 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:44.318346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.329686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d10374-ca23-4fa8-bec8-91cf2ad659ad admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.402413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.403525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.404004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.404543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.405901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.410276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.500761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1025/2063] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:44 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:44.507220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.520530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d027d-61af-494c-b334-8101555a2d02 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.526124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.526911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.527283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.527663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.528550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.530371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.548391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1039/2064] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:44.552348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.569116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a657905-d7ca-4fdb-b994-b58e3a43f036 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.575782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.577081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.577552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.577946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.579090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.581999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75b8d5-cdfb-4f9b-b0ce-c7f43416a02e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.600383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1026/2065] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:44.605910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:44.621192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b5df4-d16a-45b6-98ac-cfb3c51ac976 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:44.627108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:44.628216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.629206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.629560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] PATH_INFO: `/v3/projects/af43c72fe0cc46779ce2d66ab1b01502/users/48b5dbbc748c4b19a0456337a6495e95/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.644474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:44.659310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=af43c72fe0cc46779ce2d66ab1b01502, user_id=48b5dbbc748c4b19a0456337a6495e95, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:44.666992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:44.747745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1040/2066] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:44 2022] PUT /identity/v3/projects/af43c72fe0cc46779ce2d66ab1b01502/users/48b5dbbc748c4b19a0456337a6495e95/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:44.759393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:44.759845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:44.760356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:44.804119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] MFA Rules not processed for user `48b5dbbc748c4b19a0456337a6495e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:44.806032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:44.943985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1027/2067] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:44.948864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.040110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f29e6b-d75d-4e62-bdab-ae26929d4ebe admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.051475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.052487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.052929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.053324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.054367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.057403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.186489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1041/2068] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:44 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:45.191098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.204020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2107b97-78bd-4024-ba90-06bb1657b836 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.262716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:45.275729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.276678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.276831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.277083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.277835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.279989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.404356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1028/2069] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:45 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:45.407118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.421888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40a52-7832-4a8c-906b-67cf3b1d6859 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.427344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.428021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.428331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.428618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.429363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.431037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.444973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1042/2070] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:45.449893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.462269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f550102-c3d0-4ba6-a9c9-341976b553cd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.467425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.468574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.468882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.469172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] PATH_INFO: `/v3/projects/fbda41d50eeb48a6817a7d28736b3e11/users/d658690c291340c588382477b6448a1b/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.489220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbda41d50eeb48a6817a7d28736b3e11, user_id=d658690c291340c588382477b6448a1b, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.492380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.609069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1029/2071] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:45 2022] PUT /identity/v3/projects/fbda41d50eeb48a6817a7d28736b3e11/users/d658690c291340c588382477b6448a1b/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:45.614487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.625913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d5acb-8936-43f3-a236-f7b66fccea33 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.685943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.686541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.686831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.687113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.687867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.689533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161d5ca-9019-48d3-8313-4aa10037eeb6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.707431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1043/2072] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:45.713920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.732768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ca82-f0a0-4d7d-bd4a-2f8fbd9067ec admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:45.740688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:45.742203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.742695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.743255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] PATH_INFO: `/v3/projects/fbda41d50eeb48a6817a7d28736b3e11/users/d658690c291340c588382477b6448a1b/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.746191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbda41d50eeb48a6817a7d28736b3e11, user_id=d658690c291340c588382477b6448a1b, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:45.750696 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:45.767299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1030/2073] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:45 2022] PUT /identity/v3/projects/fbda41d50eeb48a6817a7d28736b3e11/users/d658690c291340c588382477b6448a1b/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:45.781065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:45.782183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:45.782616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:45.815352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] MFA Rules not processed for user `d658690c291340c588382477b6448a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:45.817774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:45.884402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:45.929730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1044/2074] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:45.938585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:45.940754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:46.036543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b16eb-ef61-4d3e-8e93-b2380969d0cb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:46.043473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:46.044569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:46.045038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:46.045638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:46.063525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:46.066651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a01e9-2638-4c63-9668-d584fe796d02 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:46.090208 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1031/2075] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:47.139489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:47.251026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c627980-49a5-42f0-a068-214a2d37eaff None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:47.257361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:47.258821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:47.258821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:47.259037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] PATH_INFO: `/v3/users/48b5dbbc748c4b19a0456337a6495e95` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:47.260215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b5dbbc748c4b19a0456337a6495e95)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:47.262473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:47.468258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b5dbbc748c4b19a0456337a6495e95'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:47.475939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b5dbbc748c4b19a0456337a6495e95'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:47.490437 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b5dbbc748c4b19a0456337a6495e95'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:47.491190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b5dbbc748c4b19a0456337a6495e95'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:47.491812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b5dbbc748c4b19a0456337a6495e95'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:47.561948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1045/2076] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:47 2022] DELETE /identity/v3/users/48b5dbbc748c4b19a0456337a6495e95 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:47.567180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:47.567731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:47.567844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:47.586433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:47.587674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:47.664105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1032/2077] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:47.674245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:47.676323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:47.740426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0594305-4dfb-40fe-a192-359d4aebc890 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:47.744536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:47.745206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:47.745498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:47.745782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:47.756597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:47.758094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:47.769037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1046/2078] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:48.210280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:48.213518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:48.228008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531a38d-63a2-4a66-81c9-08420d4acb18 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:48.233104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:48.234018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:48.234442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:48.234607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] PATH_INFO: `/v3/projects/af43c72fe0cc46779ce2d66ab1b01502` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:48.235906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af43c72fe0cc46779ce2d66ab1b01502)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:48.238722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:48.399030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:48.406663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af43c72fe0cc46779ce2d66ab1b01502', 'request_id': 'req-f4c09d85-ff65-4148-93ee-72a5b204f8d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.414562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af43c72fe0cc46779ce2d66ab1b01502', 'request_id': 'req-f4c09d85-ff65-4148-93ee-72a5b204f8d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.533828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af43c72fe0cc46779ce2d66ab1b01502', 'request_id': 'req-f4c09d85-ff65-4148-93ee-72a5b204f8d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.547738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] The token cache is being invalidate because project af43c72fe0cc46779ce2d66ab1b01502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:48.548243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.549058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.555718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1033/2079] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:48 2022] DELETE /identity/v3/projects/af43c72fe0cc46779ce2d66ab1b01502 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:48.557335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:48.564071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:48.568511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83dd84-67a5-4234-9ed1-ccda0d7f1933 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:48.638119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:48.639135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:48.639423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:48.639705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] PATH_INFO: `/v3/users/d658690c291340c588382477b6448a1b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:48.641103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d658690c291340c588382477b6448a1b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:48.643110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:48.935712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd658690c291340c588382477b6448a1b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.951834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd658690c291340c588382477b6448a1b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.975274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd658690c291340c588382477b6448a1b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.976069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd658690c291340c588382477b6448a1b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:48.976735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-cab20c6f-c892-4e68-a5fb-bc1d5e02aab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd658690c291340c588382477b6448a1b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:49.108760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1047/2080] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:48 2022] DELETE /identity/v3/users/d658690c291340c588382477b6448a1b => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:49.696061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:49.698445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:49.721651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c09d85-ff65-4148-93ee-72a5b204f8d0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:49.797369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:49.798422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:49.798804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:49.799174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] PATH_INFO: `/v3/projects/fbda41d50eeb48a6817a7d28736b3e11` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:49.801614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbda41d50eeb48a6817a7d28736b3e11)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:49.804804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:49.932530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbda41d50eeb48a6817a7d28736b3e11', 'request_id': 'req-532a24c1-3ad4-47ce-b140-5880a7481441'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:49.943088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbda41d50eeb48a6817a7d28736b3e11', 'request_id': 'req-532a24c1-3ad4-47ce-b140-5880a7481441'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:50.036419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbda41d50eeb48a6817a7d28736b3e11', 'request_id': 'req-532a24c1-3ad4-47ce-b140-5880a7481441'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:50.048482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] The token cache is being invalidate because project fbda41d50eeb48a6817a7d28736b3e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:50.048834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:50.049686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-532a24c1-3ad4-47ce-b140-5880a7481441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:50.051479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1034/2081] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:49 2022] DELETE /identity/v3/projects/fbda41d50eeb48a6817a7d28736b3e11 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:50.056633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.057474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.057662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.077704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:50.079137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:50.149344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1048/2082] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:50.157336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.157633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.157918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.179020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:50.180383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:50.188743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1035/2083] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:50.193959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.204842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba711c76-9c6b-4db0-9028-bcdf5aed548c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.211446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.212157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.212480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.215121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.215121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:50.215121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.215998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.231304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1049/2084] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:50.237242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.251682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b1b73-3f18-4d80-b1a8-2207674b419c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.257088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.257741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.258035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.258322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.259034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.261042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.392498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1036/2085] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:50.397270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.408407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cfcd9-8cd0-4836-8d0e-4347c3b687bf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.424151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:50.504395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.505063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.505410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.505796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.506778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.508993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.586179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1050/2086] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/users => generated 284 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:50.590844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.602162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b3639-c831-4de6-99ef-f58255c1b2cb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.611332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.612285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.612667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.613256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.614174 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.616281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.630867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1037/2087] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:50.638502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.650283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4d4a6-50ed-429e-9d1d-e7f026766343 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.655254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.661875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.661875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.661875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.661875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.661875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd505e1-720a-4620-b49d-0972b38dd0ed admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.670431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1051/2088] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:50.676024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.694668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1c9b8-67fe-4092-9eb1-ac3e715e3961 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.700752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.702725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.703116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.703503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] PATH_INFO: `/v3/projects/14e79c8ff01b471e9ad2c7ce6fd18f89/users/c82aa5650f2b47609c8fa1991212b0d0/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.723335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e79c8ff01b471e9ad2c7ce6fd18f89, user_id=c82aa5650f2b47609c8fa1991212b0d0, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.726185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.729706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:50.810058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1038/2089] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:50 2022] PUT /identity/v3/projects/14e79c8ff01b471e9ad2c7ce6fd18f89/users/c82aa5650f2b47609c8fa1991212b0d0/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:50.817751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.818070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.818356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.819071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.821060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:50.851143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] MFA Rules not processed for user `c82aa5650f2b47609c8fa1991212b0d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:50.852587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:50.922544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fd6c0-0b81-4719-945c-1933c716c375 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.931965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.933575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.934065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.934558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] PATH_INFO: `/v3/users/17b7e7c217174e318b0ab669a94eacf7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.936766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b7e7c217174e318b0ab669a94eacf7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.940590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:50.942953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1052/2090] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:50.947626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:50.959641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5de403-8b35-489b-9527-bc77a0eee142 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:50.974240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:50.975403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:50.975926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:50.976466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:50.977930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:50.981747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.100828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1053/2091] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:50 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:51.105428 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.166612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:51.170434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf38836-6e42-4d64-b340-980435763d48 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.175225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.176691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.176985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.177346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.178324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.180283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9bb17-4c46-48b1-ab8f-fd2d72c3484a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.196556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b7e7c217174e318b0ab669a94eacf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:51.197278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b7e7c217174e318b0ab669a94eacf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:51.213039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b7e7c217174e318b0ab669a94eacf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:51.220213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b7e7c217174e318b0ab669a94eacf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:51.304716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b7e7c217174e318b0ab669a94eacf7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:51.312008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1039/2093] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:50 2022] DELETE /identity/v3/users/17b7e7c217174e318b0ab669a94eacf7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:51.312241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1054/2093] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:51 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:51.316375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.316983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.317490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.317809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.318312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:51.335979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:51.337263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:51.396065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ef34d-8515-4a15-858b-a4a1b436be02 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.401940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.402844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.403313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.403779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.404954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.406067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1055/2094] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:51.408430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.416911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.418432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:51.426358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1040/2095] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:51.430763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.441706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0768f-b70c-496e-aefd-291fd309a3cf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.446905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.447837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.448117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.448555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] PATH_INFO: `/v3/projects/058ce268c386430c8b5c0f6cb5884313/users/cd4e23d4011d4e5ca5433969f4e59ae0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.461454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:51.469194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058ce268c386430c8b5c0f6cb5884313, user_id=cd4e23d4011d4e5ca5433969f4e59ae0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.471972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.506198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd101f-af02-465a-baf0-2cb12556f26c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.510007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.510670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.510973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.511252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.522067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.523539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.535256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1056/2096] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:51.586842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1041/2097] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:51 2022] PUT /identity/v3/projects/058ce268c386430c8b5c0f6cb5884313/users/cd4e23d4011d4e5ca5433969f4e59ae0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:51.592778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.613943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cca3-4f20-4eac-8e6c-f32a40089509 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.700195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.700912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.701204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.701494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.702246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.703891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92461f14-11a9-4152-bccf-af99086c9688 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.715715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1057/2098] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:51.726371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.735761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61285-bb9e-4ac5-b102-ed0afb4c3c01 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:51.742302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:51.743997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.745123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.745856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] PATH_INFO: `/v3/projects/058ce268c386430c8b5c0f6cb5884313/users/cd4e23d4011d4e5ca5433969f4e59ae0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.750227 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058ce268c386430c8b5c0f6cb5884313, user_id=cd4e23d4011d4e5ca5433969f4e59ae0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:51.757359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:51.779323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1042/2099] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:51 2022] PUT /identity/v3/projects/058ce268c386430c8b5c0f6cb5884313/users/cd4e23d4011d4e5ca5433969f4e59ae0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:51.799695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:51.800114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:51.801112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:51.846990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] MFA Rules not processed for user `cd4e23d4011d4e5ca5433969f4e59ae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:51.849818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:51.870987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:51.963229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1058/2100] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:51.979730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:51.982653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:52.105850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6eabe3-26b5-43d9-8ae4-5df3c4340886 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:52.111953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:52.114053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:52.114427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:52.114861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:52.130096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:52.132750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:52.153476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:52.155602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:52.162220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1043/2101] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:52.278197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06dba3-089f-46b3-af84-c0c56e449696 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:52.283693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:52.285410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:52.285803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:52.286235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] PATH_INFO: `/v3/projects/41512c19609541879690cbc5cfc963e1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:52.287729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] RBAC: Authorizing `identity:delete_project(project_id=41512c19609541879690cbc5cfc963e1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:52.289973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:52.494666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41512c19609541879690cbc5cfc963e1', 'request_id': 'req-467c34b3-8208-4561-a452-b398c52a423f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.505151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41512c19609541879690cbc5cfc963e1', 'request_id': 'req-467c34b3-8208-4561-a452-b398c52a423f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.567550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41512c19609541879690cbc5cfc963e1', 'request_id': 'req-467c34b3-8208-4561-a452-b398c52a423f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.595030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] The token cache is being invalidate because project 41512c19609541879690cbc5cfc963e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:52.595518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.596557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.599092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1059/2102] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:52 2022] DELETE /identity/v3/projects/41512c19609541879690cbc5cfc963e1 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:52.608559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:52.621500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e528c-7413-4659-bde1-ad44dfe45bb2 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:52.688731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:52.689651 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:52.689919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:52.690200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] PATH_INFO: `/v3/users/87ecf9d8bc454fc28beb9510a0e553bb` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:52.694072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ecf9d8bc454fc28beb9510a0e553bb)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:52.696147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:52.931632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ecf9d8bc454fc28beb9510a0e553bb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.932659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ecf9d8bc454fc28beb9510a0e553bb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.948663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ecf9d8bc454fc28beb9510a0e553bb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.957026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ecf9d8bc454fc28beb9510a0e553bb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:52.961159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:53.043228 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a7863fcc-d5fe-4b00-bb64-6adb8a735f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ecf9d8bc454fc28beb9510a0e553bb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:53.058676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1044/2103] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:52 2022] DELETE /identity/v3/users/87ecf9d8bc454fc28beb9510a0e553bb => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:53.674034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:53.676049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:53.692615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c34b3-8208-4561-a452-b398c52a423f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:53.743953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:53.763938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:53.764892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:53.765251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:53.765541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] PATH_INFO: `/v3/projects/15b8701433d2402682d6b02d6313040c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:53.766881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b8701433d2402682d6b02d6313040c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:53.768940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:53.897618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b8701433d2402682d6b02d6313040c', 'request_id': 'req-0baf240d-cbe3-48e6-8c50-12bc8f549078'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:53.904369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b8701433d2402682d6b02d6313040c', 'request_id': 'req-0baf240d-cbe3-48e6-8c50-12bc8f549078'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:54.026109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b8701433d2402682d6b02d6313040c', 'request_id': 'req-0baf240d-cbe3-48e6-8c50-12bc8f549078'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:54.044464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] The token cache is being invalidate because project 15b8701433d2402682d6b02d6313040c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:54.044969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:54.045904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0baf240d-cbe3-48e6-8c50-12bc8f549078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:54.047653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1060/2104] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:53 2022] DELETE /identity/v3/projects/15b8701433d2402682d6b02d6313040c => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:54.053763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.054299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.054412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.075656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:54.077005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:54.148659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1045/2105] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:54.163146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.163677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.164130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.188430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:54.189793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:54.201944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1061/2106] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:54.207050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.218242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a11530-a70f-4910-9438-e8996cedcd83 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.224156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.224993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.225319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.225640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.226521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:54.226966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.229015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:54.246658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1046/2107] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:54.252187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.267370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f6c2a-13a3-4bf6-818e-158fd7e293b6 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.274247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.275160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.275539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.275931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.276851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.279050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:54.428813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1062/2108] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:54 2022] POST /identity/v3/projects => generated 399 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:54.432356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.461684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a265125-cce6-45ff-8437-117097cb73bb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.576883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.580171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.580171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.580666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.582905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.590339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:54.720135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1047/2109] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:54 2022] POST /identity/v3/users => generated 300 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:54.726012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.748022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd55e2-903d-43ff-bd45-1872623707c6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.767621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.769607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.770439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.771447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.774123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.788827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:54.802678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1063/2110] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:54.817684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.830492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f6c78-04c6-43b0-8700-bdec1c105d41 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.844692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.846756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.848529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.850040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.851951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.855456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09b4c8-0c82-4096-b851-a11cdbe5bc63 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:54.871887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1048/2111] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:54.877403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:54.896902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2469d-d778-4621-a748-4df0e00f580c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:54.904947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:54.906531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:54.907065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:54.907585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] PATH_INFO: `/v3/projects/2db99b517c914d0fa55ec1e2eeeaffe7/users/11b23ae7d0b640068761686c31813dad/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:54.932375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db99b517c914d0fa55ec1e2eeeaffe7, user_id=11b23ae7d0b640068761686c31813dad, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:54.935104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:55.040928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1064/2112] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:54 2022] PUT /identity/v3/projects/2db99b517c914d0fa55ec1e2eeeaffe7/users/11b23ae7d0b640068761686c31813dad/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:55.067120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:55.068542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:55.070146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:55.119948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] MFA Rules not processed for user `11b23ae7d0b640068761686c31813dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:55.122139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:55.213701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:55.243966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1049/2113] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:26:55.251868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:55.347531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56849c5e-a77b-4b0b-8f2c-c9a691137641 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:55.353219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:55.353914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:55.354461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:55.354874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:55.355722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:55.357840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:55.375126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:55.475360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1065/2114] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:55 2022] POST /identity/v3/projects => generated 401 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:55.480642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:55.501522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd1c1d-63d1-4852-ac41-90ae87b9d26d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:55.597447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:55.598357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:55.598779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:55.599178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:55.600032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:55.602226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:55.727258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1050/2115] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:55 2022] POST /identity/v3/users => generated 307 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:26:55.729375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:55.748045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19acce3-dbd1-4a04-9419-1e1e847524de admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:55.756410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:55.761993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:55.761993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:55.761993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:55.761993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:55.761993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:55.781333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1066/2116] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:26:55.790534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:55.809870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27154572-9fff-4860-afb6-1fd89abf7d39 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:55.818485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:55.820097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:55.825063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:55.825063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] PATH_INFO: `/v3/projects/293f2f4b3759491bb5f16cefb6dd375d/users/74a7e171cd6e43e08b1785fee8545598/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:55.853807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293f2f4b3759491bb5f16cefb6dd375d, user_id=74a7e171cd6e43e08b1785fee8545598, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:55.859525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:55.959865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1051/2117] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:55 2022] PUT /identity/v3/projects/293f2f4b3759491bb5f16cefb6dd375d/users/74a7e171cd6e43e08b1785fee8545598/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:55.964717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:55.977309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6375b-689a-478d-820e-04bfe3e4d5ec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.080409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.081568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.082527 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.083152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.084571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.087588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2df22-8500-4d17-ab1b-a21151ff0c2b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.107551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1067/2118] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:56.115542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:56.132856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d5ad3-82ac-4495-9f4d-68bbdd83b838 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.140456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.142009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.142552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.146309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] PATH_INFO: `/v3/projects/293f2f4b3759491bb5f16cefb6dd375d/users/74a7e171cd6e43e08b1785fee8545598/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.147663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] RBAC: Authorizing `identity:create_grant(project_id=293f2f4b3759491bb5f16cefb6dd375d, user_id=74a7e171cd6e43e08b1785fee8545598, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.152796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.172536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1052/2119] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:56 2022] PUT /identity/v3/projects/293f2f4b3759491bb5f16cefb6dd375d/users/74a7e171cd6e43e08b1785fee8545598/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:56.185081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.185413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.186578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.227675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] MFA Rules not processed for user `74a7e171cd6e43e08b1785fee8545598`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:56.231189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:56.343691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1068/2120] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:56.362544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:56.366232 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:56.367104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:56.368943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:56.475502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468af899-199e-4445-b2be-7c2451ade0b5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.480127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.480873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.481194 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.481452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.492879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.494375 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.506775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1069/2121] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:56 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:56.533811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4b7df-4bf4-4f1c-a5a0-e52b2a6c996f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.540822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.542676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.543253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.543831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] PATH_INFO: `/v3/users/515504022e2c4e6faa46f6b68c58c438` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.546257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=515504022e2c4e6faa46f6b68c58c438)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.550083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.760606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:56.771253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943e6d-d3b8-451d-9d6a-34907d9a63cf service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.775221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.779813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.779813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.779813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.781838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:56.808893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:56.811027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '515504022e2c4e6faa46f6b68c58c438'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:56.812040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '515504022e2c4e6faa46f6b68c58c438'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:56.839827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '515504022e2c4e6faa46f6b68c58c438'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:56.854614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '515504022e2c4e6faa46f6b68c58c438'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:56.868293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.869803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.886299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1070/2122] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:26:56.887608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:56.889467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:56.904680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0d7ab-b2fa-427f-9789-83af422e2fbf service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:56.911987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:56.912923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.913212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.913494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] PATH_INFO: `/v3/users/c82aa5650f2b47609c8fa1991212b0d0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:56.917580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82aa5650f2b47609c8fa1991212b0d0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:56.917580 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:56.951865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7ee85d10-1c21-4b2f-a312-caf541507d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '515504022e2c4e6faa46f6b68c58c438'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:56.963200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1053/2123] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:56 2022] DELETE /identity/v3/users/515504022e2c4e6faa46f6b68c58c438 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:56.968140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:56.969176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:56.969510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:57.004346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:57.004346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:57.114275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1054/2124] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:57.126395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:57.129391 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:57.200525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82aa5650f2b47609c8fa1991212b0d0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:57.227793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82aa5650f2b47609c8fa1991212b0d0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:57.250844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048be9b-9f8c-41f3-b13b-5133e89d4b23 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:57.255540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82aa5650f2b47609c8fa1991212b0d0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:57.256810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82aa5650f2b47609c8fa1991212b0d0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:57.257715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82aa5650f2b47609c8fa1991212b0d0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:57.261317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:57.262496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:57.262986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:57.263475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:57.266793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:57.285851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:57.288150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6cbca-5e87-464e-8c6f-3024b16c12d7 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:57.307136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1055/2125] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:57.363701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1071/2126] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:56 2022] DELETE /identity/v3/users/c82aa5650f2b47609c8fa1991212b0d0 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:57.368597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:57.370394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:57.370394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:57.390894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:57.392140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:57.496191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1056/2127] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:57.509181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:57.511344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:57.592927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4d7a9-a546-4557-9d42-b9b3690cf359 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:57.598264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:57.599215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:57.599720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:57.600434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:57.614165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:57.614915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:57.629723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1072/2128] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:26:58.107960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.110303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.136114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640515cb-9428-44dd-ab89-9547199a8b0f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.147799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.149117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.149797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.150494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] PATH_INFO: `/v3/projects/87a2c6be97a44fdd99e20e50a791da2b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.152952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a2c6be97a44fdd99e20e50a791da2b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.155573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:58.158080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:58.294614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a2c6be97a44fdd99e20e50a791da2b', 'request_id': 'req-795d9dd5-681d-4157-b5f2-d95faba472cd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.303200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a2c6be97a44fdd99e20e50a791da2b', 'request_id': 'req-795d9dd5-681d-4157-b5f2-d95faba472cd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.315606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.318826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.337785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d17cb1-e01f-4b9f-b4e5-48dc2a73f8ba service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.366306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:58.396674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a2c6be97a44fdd99e20e50a791da2b', 'request_id': 'req-795d9dd5-681d-4157-b5f2-d95faba472cd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.410611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] The token cache is being invalidate because project 87a2c6be97a44fdd99e20e50a791da2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:58.411031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.412249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-795d9dd5-681d-4157-b5f2-d95faba472cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.414285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1057/2129] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:58 2022] DELETE /identity/v3/projects/87a2c6be97a44fdd99e20e50a791da2b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:58.420623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.421312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763eee3-e3b0-4895-908e-d74f092ac789 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.421479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.421656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763eee3-e3b0-4895-908e-d74f092ac789 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.421801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.422110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] PATH_INFO: `/v3/projects/14e79c8ff01b471e9ad2c7ce6fd18f89` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.422359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763eee3-e3b0-4895-908e-d74f092ac789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.423584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e79c8ff01b471e9ad2c7ce6fd18f89)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.426140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:58.447683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e763eee3-e3b0-4895-908e-d74f092ac789 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:58.449665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e763eee3-e3b0-4895-908e-d74f092ac789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.464677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1058/2130] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:58.477178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.477388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.477789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.503062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:58.505988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.522200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1059/2131] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:58.528435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.539674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e924bcf-7ff6-4763-93d8-c84a37d27654 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.546372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.547749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.548218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.548840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.553050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:26:58.553620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.556576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:58.573678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e79c8ff01b471e9ad2c7ce6fd18f89', 'request_id': 'req-d6f7310b-8396-4fdd-92cb-0162e964881f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.578353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1060/2132] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:26:58.584828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.591097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e79c8ff01b471e9ad2c7ce6fd18f89', 'request_id': 'req-d6f7310b-8396-4fdd-92cb-0162e964881f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.595952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f913934-e0a1-431e-955d-a9a154fd5bab admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.703322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.704134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.704755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.705110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.707253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.710876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:58.726917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e79c8ff01b471e9ad2c7ce6fd18f89', 'request_id': 'req-d6f7310b-8396-4fdd-92cb-0162e964881f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.746635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] The token cache is being invalidate because project 14e79c8ff01b471e9ad2c7ce6fd18f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:26:58.747143 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.748160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:58.750227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1073/2133] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:58 2022] DELETE /identity/v3/projects/14e79c8ff01b471e9ad2c7ce6fd18f89 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:58.755072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.756915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.774536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7310b-8396-4fdd-92cb-0162e964881f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.779997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.781424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.781882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.782303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] PATH_INFO: `/v3/users/cd4e23d4011d4e5ca5433969f4e59ae0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.784014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4e23d4011d4e5ca5433969f4e59ae0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.787278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:58.829821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1061/2134] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:58 2022] POST /identity/v3/projects => generated 375 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:58.835158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:58.837790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:58.952018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90e12-bc08-4176-90a8-fc15973ec7b6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:58.960862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:58.961970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:58.962462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:58.962949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:58.963988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:58.967790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:59.064342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4e23d4011d4e5ca5433969f4e59ae0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:59.067043 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1062/2135] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:58 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:59.067725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4e23d4011d4e5ca5433969f4e59ae0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:59.074222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:59.086767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4e23d4011d4e5ca5433969f4e59ae0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:59.087832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4e23d4011d4e5ca5433969f4e59ae0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:59.088569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4e23d4011d4e5ca5433969f4e59ae0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:26:59.090631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0f34d-829a-40b5-a3fd-2dde4bc3da55 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:59.098743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:59.099669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:59.100124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:59.100662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:59.101885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:59.103914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:59.118092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1063/2136] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:59.122336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:59.124221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:59.140430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d6c9-bc56-4abe-b53e-b86b9e67ba9e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:59.146508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:59.147262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:59.147594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:59.147930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:59.149105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:59.151153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0632cb-2b04-4924-a5ad-a13bbf081c33 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:59.157514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1074/2137] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:26:58 2022] DELETE /identity/v3/users/cd4e23d4011d4e5ca5433969f4e59ae0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:26:59.170244 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1064/2138] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:26:59.172512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:59.221498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:59.276709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c627192-016a-450d-a03c-508f77f5480b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:59.280103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:26:59.286575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:59.288208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:59.288771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:59.289285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] PATH_INFO: `/v3/projects/73ac0cda9f374fc6ba89fa8ebacbe603/users/77db1d94ede442f68e55ef7b08a9c2d9/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:59.321166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ac0cda9f374fc6ba89fa8ebacbe603, user_id=77db1d94ede442f68e55ef7b08a9c2d9, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:59.327356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:59.453898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1075/2139] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:26:59 2022] PUT /identity/v3/projects/73ac0cda9f374fc6ba89fa8ebacbe603/users/77db1d94ede442f68e55ef7b08a9c2d9/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:26:59.473574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:59.474083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:59.474083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:59.511859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] MFA Rules not processed for user `77db1d94ede442f68e55ef7b08a9c2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:26:59.513309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:59.603568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1065/2140] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:26:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:26:59.610838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:59.769946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747e52e-4ea4-475f-9022-8238e8500834 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:59.783204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:26:59.783204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:26:59.783204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:26:59.783204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:26:59.795767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:26:59.795767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:26:59.893543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:26:59.896341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:26:59.940130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01388f3f-fde0-45e3-b2d1-c131ba5e7965 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:26:59.944578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1076/2141] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:26:59 2022] POST /identity/v3/projects => generated 377 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:26:59.946558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.056071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39473e-eead-4e8c-95a9-a5a279ca1070 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.062032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.063333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.063630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.063917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] PATH_INFO: `/v3/projects/058ce268c386430c8b5c0f6cb5884313` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.065322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058ce268c386430c8b5c0f6cb5884313)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.067853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.069095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.069516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.069683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.070098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.071466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.075768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.192013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1077/2142] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:26:59 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:00.196499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.208573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d841-aed5-490b-a672-37572cabda46 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.218226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.219360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.219871 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.220568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.222230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.224359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.235134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:00.238710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058ce268c386430c8b5c0f6cb5884313', 'request_id': 'req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:00.245024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1078/2143] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:00.246672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.248168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058ce268c386430c8b5c0f6cb5884313', 'request_id': 'req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:00.259633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ce7b1-f1df-4a7a-9836-27866792465a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.265408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:00.322404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058ce268c386430c8b5c0f6cb5884313', 'request_id': 'req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:00.332451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.333586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.334392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.334392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.335083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.336961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564673c1-1be6-418c-a209-149e5fecd5de admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.339154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] The token cache is being invalidate because project 058ce268c386430c8b5c0f6cb5884313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:00.339624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:00.340618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:00.343236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1066/2144] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:26:59 2022] DELETE /identity/v3/projects/058ce268c386430c8b5c0f6cb5884313 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:00.350811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1079/2145] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:00.355733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.357742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c66-3766-49f0-998b-06c1c62edb09 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.358037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c66-3766-49f0-998b-06c1c62edb09 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.358321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c66-3766-49f0-998b-06c1c62edb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.362786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:00.385626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-81772c66-3766-49f0-998b-06c1c62edb09 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:00.387070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fea4b7-f54a-4fee-9dc2-1e5076b6e6d9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.387260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-81772c66-3766-49f0-998b-06c1c62edb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:00.393390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.395097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.395525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.395976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] PATH_INFO: `/v3/projects/92dfe44cdafe4286b2cd172f5464812b/users/60e65e0617c64761a7929aca27b4f45c/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.397681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1080/2146] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:00.405884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.406189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.406451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.416706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dfe44cdafe4286b2cd172f5464812b, user_id=60e65e0617c64761a7929aca27b4f45c, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.419527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c29ea-997d-4527-a173-2f0617ae8789 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.428812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:00.430969 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:00.444578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1081/2147] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:00.451042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.461852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300367-4eee-4700-b5ec-23ea14b79c43 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.466710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.467508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.467882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.468190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.469192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:00.469544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.471227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.485271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1082/2148] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:00.491446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.502245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe858-50a1-48d8-9cc9-b3bbd38bfd65 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.510017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1067/2149] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:00 2022] PUT /identity/v3/projects/92dfe44cdafe4286b2cd172f5464812b/users/60e65e0617c64761a7929aca27b4f45c/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:00.530153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.530632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.531186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.579124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] MFA Rules not processed for user `60e65e0617c64761a7929aca27b4f45c`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:00.580781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:00.590816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.591834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.592193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.592575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.593416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.595733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.668862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1068/2150] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:00.676441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.688873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307c6b3-f85a-414a-91fb-8ed0f338dde2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.695172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.696087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.696573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.697019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.698290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.703022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:00.728931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1083/2151] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:00.734011 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.832504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ca729-c31b-403a-8eec-ec5f992b2cd1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:00.839290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1069/2152] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:00.840885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:00.844677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:00.957945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:00.959042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:00.959448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:00.959908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:00.960955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:00.963249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.014490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840fe9c-c6bb-4f2c-8e90-c9fad307e42c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.020093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.020898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.021186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.021468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.022181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.024968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.047320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1084/2153] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/users => generated 288 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:01.053995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.065798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b8f57-9579-4eba-be41-4c89751f2a15 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.080942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.081487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.082335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.083359 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.086124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.087863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.102799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1085/2154] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:01.108161 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.119125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1070/2155] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:00 2022] POST /identity/v3/users => generated 294 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:01.120463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.132907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50f0ae-bc75-463d-972d-2186ea143a38 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.133131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0ebeb-ee82-451e-a9a5-d1f96c66a864 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.142718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.143794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.144846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.145330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.146831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.150159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.157185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.160407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.160407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.161290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.162880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.167525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.173679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1086/2156] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:01.173888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:01.175279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.197805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1071/2157] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:01.198009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d70ce9-09e7-435a-a858-1f5ac93810df admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.200150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.204939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.206752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.207337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.207933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] PATH_INFO: `/v3/projects/f716a9a939ba4ecea613eddc030a65d1/users/c9543123d9554d8e884cf1339ca959a0/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.221162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd8c15-076c-4954-bc5d-fdbae019d612 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.232186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.233893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.235840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.237787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] PATH_INFO: `/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61/users/edb0f2660167457a921ecbc82de6bfc6/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.240588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f716a9a939ba4ecea613eddc030a65d1, user_id=c9543123d9554d8e884cf1339ca959a0, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.249708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577aa70b-4333-48ac-bce2-d316251cde61 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.275345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aeaf6fc849b4a29b1bec0994e8add61, user_id=edb0f2660167457a921ecbc82de6bfc6, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.278658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.359953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1087/2158] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:01 2022] PUT /identity/v3/projects/f716a9a939ba4ecea613eddc030a65d1/users/c9543123d9554d8e884cf1339ca959a0/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:01.364938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.365360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.365779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.408988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1072/2159] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:01 2022] PUT /identity/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61/users/edb0f2660167457a921ecbc82de6bfc6/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:01.417700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.423410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] MFA Rules not processed for user `c9543123d9554d8e884cf1339ca959a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:01.425037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:01.529673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1088/2160] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:01.536984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.539877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b16ba-0d3b-4dc3-aaea-b066b62a60ea admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.546736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.547599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.547977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.548412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.549294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.551250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1067c35-ab24-40d2-bf8e-e5360add079c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.551451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.557427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.558253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.558666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.559120 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.560052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.562595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.568589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1073/2161] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:01.571148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.584226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f54a51-f902-471b-a050-413cf8ea891f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.589524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.590447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.590758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.596639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] PATH_INFO: `/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61/users/edb0f2660167457a921ecbc82de6bfc6/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.596639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aeaf6fc849b4a29b1bec0994e8add61, user_id=edb0f2660167457a921ecbc82de6bfc6, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.596639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0ca4cb-3eb9-44e5-86d6-23e9ab346863 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.615569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1074/2162] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:01 2022] PUT /identity/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61/users/edb0f2660167457a921ecbc82de6bfc6/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:01.628034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.629283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.629789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.677698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] MFA Rules not processed for user `edb0f2660167457a921ecbc82de6bfc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:01.679816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:01.701030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1089/2163] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:01 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:01.705837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.749273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:01.796415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1075/2164] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:01.822343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40572c47-1f0a-4b14-a59a-465121028498 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.825274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.827454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:01.833224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.834268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.834649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.835139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.836040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.838305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:01.887100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:01.903900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d7577-5b30-4722-92ea-011e082c3871 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.965711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1090/2165] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:01 2022] POST /identity/v3/users => generated 295 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:01.970037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:01.974145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.975032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.975422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.975991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.978180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:01.981457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863e27c-7af3-4401-b9b9-cdb6362c30c8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:01.986471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:01.987162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:01.990983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:01.990983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:01.990983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:01.990983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.011223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1091/2166] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:02.012348 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:02.034109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311d056-7eae-49ed-87fc-949693d3d6b4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:02.042423 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:02.043773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:02.044228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:02.044651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] PATH_INFO: `/v3/projects/ebd6db36dfef4e41a6c5032d1df88541/users/6ea46ebf71ff474fbf70b4774b684415/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:02.068850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd6db36dfef4e41a6c5032d1df88541, user_id=6ea46ebf71ff474fbf70b4774b684415, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:02.071989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.075396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:02.078254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.090048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1076/2167] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:27:02.157818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1092/2168] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:02 2022] PUT /identity/v3/projects/ebd6db36dfef4e41a6c5032d1df88541/users/6ea46ebf71ff474fbf70b4774b684415/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:02.163242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:02.183837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810e408-b205-443e-9e7a-f4049adf9bfe service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:02.276368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:02.277458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:02.277904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:02.278414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:02.279475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:02.282791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2025536b-4d46-41ba-ac66-d1996af1f64e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.299280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1077/2169] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:02.304659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:02.317833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed80d3-6c9d-4dbf-bc0c-e3aafbc6a4cb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:02.323400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:02.327729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:02.327729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:02.327729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] PATH_INFO: `/v3/projects/ebd6db36dfef4e41a6c5032d1df88541/users/6ea46ebf71ff474fbf70b4774b684415/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:02.327729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd6db36dfef4e41a6c5032d1df88541, user_id=6ea46ebf71ff474fbf70b4774b684415, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:02.330181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.343105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1093/2170] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:02 2022] PUT /identity/v3/projects/ebd6db36dfef4e41a6c5032d1df88541/users/6ea46ebf71ff474fbf70b4774b684415/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:02.356179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:02.356800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:02.357346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:02.410914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] MFA Rules not processed for user `6ea46ebf71ff474fbf70b4774b684415`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:02.413201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:02.506975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:02.559479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1078/2171] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:02.633923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:02.636876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:02.757015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de6bd5-91db-45d3-80c2-8cd8587f383b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:02.761148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:02.761835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:02.762123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:02.762410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:02.837625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:02.839010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa092c4-8346-46c7-9941-86ec92e224aa service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:02.849311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1094/2172] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:03.293842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:03.295741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:03.379672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19fd0f-340a-4b56-a6cb-138ff6cc37c8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:03.386928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c991c-beaa-4521-929a-9484727670da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:03.388182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c991c-beaa-4521-929a-9484727670da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:03.388577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c991c-beaa-4521-929a-9484727670da admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:03.388831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c991c-beaa-4521-929a-9484727670da admin admin] PATH_INFO: `/v3/users/11b23ae7d0b640068761686c31813dad` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:03.390274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c991c-beaa-4521-929a-9484727670da admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b23ae7d0b640068761686c31813dad)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:03.392498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c991c-beaa-4521-929a-9484727670da admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:03.619806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b23ae7d0b640068761686c31813dad'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:03.621579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b23ae7d0b640068761686c31813dad'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:03.657099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b23ae7d0b640068761686c31813dad'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:03.668116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b23ae7d0b640068761686c31813dad'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:03.779473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b23ae7d0b640068761686c31813dad'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:03.786468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1079/2173] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:03 2022] DELETE /identity/v3/users/11b23ae7d0b640068761686c31813dad => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:03.791343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:03.791880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:03.791880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:03.814848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:03.817040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:03.831645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:03.900740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1095/2174] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:03.911370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:03.913265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:03.998446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c991c-beaa-4521-929a-9484727670da admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:04.003306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:04.004614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:04.004942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:04.005343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:04.014541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:04.017570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:04.019101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:04.034437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1080/2175] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:04.468748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:04.472145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:04.561200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db204153-ac14-4c51-8a84-d6015e4206ef None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:04.565420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:04.566691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:04.567666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:04.567970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:04.570014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:04.620199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:04.621699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:04.638484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e113b1-a700-471b-b936-f8a2a836f833 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:04.657688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:04.658910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:04.659496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:04.660718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] PATH_INFO: `/v3/projects/2db99b517c914d0fa55ec1e2eeeaffe7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:04.663517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db99b517c914d0fa55ec1e2eeeaffe7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:04.663991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:04.665694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:04.667947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:04.683055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1096/2176] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:04.823147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db99b517c914d0fa55ec1e2eeeaffe7', 'request_id': 'req-65107d40-605a-4aba-8b16-efcbe15791db'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:04.833942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db99b517c914d0fa55ec1e2eeeaffe7', 'request_id': 'req-65107d40-605a-4aba-8b16-efcbe15791db'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:04.928657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db99b517c914d0fa55ec1e2eeeaffe7', 'request_id': 'req-65107d40-605a-4aba-8b16-efcbe15791db'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:04.955261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] The token cache is being invalidate because project 2db99b517c914d0fa55ec1e2eeeaffe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:04.955764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:04.956458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:04.958580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1081/2177] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:04 2022] DELETE /identity/v3/projects/2db99b517c914d0fa55ec1e2eeeaffe7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:04.963505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:04.981287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720ebb6-cee6-498e-a384-f048d840e29e service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:05.058128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:05.059036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:05.059327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:05.059612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] PATH_INFO: `/v3/users/74a7e171cd6e43e08b1785fee8545598` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:05.060987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a7e171cd6e43e08b1785fee8545598)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:05.063073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:05.083279 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:05.249753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a7e171cd6e43e08b1785fee8545598'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:05.264599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a7e171cd6e43e08b1785fee8545598'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:05.290980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a7e171cd6e43e08b1785fee8545598'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:05.296436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a7e171cd6e43e08b1785fee8545598'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:05.298414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9ab3b35-6d10-47ae-b6a5-d99a8e2dc034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a7e171cd6e43e08b1785fee8545598'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:05.412371 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1097/2178] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:04 2022] DELETE /identity/v3/users/74a7e171cd6e43e08b1785fee8545598 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:06.284722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:06.287939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:06.308708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65107d40-605a-4aba-8b16-efcbe15791db admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:06.461799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:06.463405 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:06.465058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:06.468556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] PATH_INFO: `/v3/projects/293f2f4b3759491bb5f16cefb6dd375d` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:06.474530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=293f2f4b3759491bb5f16cefb6dd375d)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:06.477594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:06.616417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293f2f4b3759491bb5f16cefb6dd375d', 'request_id': 'req-08296b00-1860-4038-8b68-93a2fff6c59c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:06.624031 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293f2f4b3759491bb5f16cefb6dd375d', 'request_id': 'req-08296b00-1860-4038-8b68-93a2fff6c59c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:06.723053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293f2f4b3759491bb5f16cefb6dd375d', 'request_id': 'req-08296b00-1860-4038-8b68-93a2fff6c59c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:06.739911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] The token cache is being invalidate because project 293f2f4b3759491bb5f16cefb6dd375d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:06.740411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:06.741290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-08296b00-1860-4038-8b68-93a2fff6c59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:06.743034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1082/2179] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:06 2022] DELETE /identity/v3/projects/293f2f4b3759491bb5f16cefb6dd375d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:06.750557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:06.751324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:06.751691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:06.773244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:06.774503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:06.889116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1098/2180] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:06.898983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:06.901611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:06.901611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:06.933958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:06.935909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:06.950415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1083/2181] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:06.955249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:06.972813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2c624-a930-4502-8741-481fdb91d2c5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:06.978569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:06.979450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:06.979978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:06.980269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:06.981072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:06.981391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:06.983148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.000281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1099/2182] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:07.005873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.014165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:07.022226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d93ffc-8143-45e4-9e46-d394d83d4949 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.029548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.035692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.035692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.035692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.035692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.035692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.126918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1084/2183] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:07 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:07.133235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.147515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a0ee6-c86d-435e-8ad3-8a2b3eac1721 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.236007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.236827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.237304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.237791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.238824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.241481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.348646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1100/2184] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:07 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:07.354401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.377392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5da8b-0856-40bd-9b5b-bec86dc103ff admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.388785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.389190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.389586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.390097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.391803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.401030 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.414310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1085/2185] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:07.419450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.435922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df78ee-dccc-4820-aa42-c382de412ae5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.441935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.442737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.443156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.443513 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.444392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.446153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f744e-23f3-46cc-bc0b-766ae9b11f5c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.448830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:07.459262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1101/2186] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:07.463428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.492954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfe8b-3d37-4ded-8aea-9bfc9f7a89cc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.494479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.495185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.495457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.495745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] PATH_INFO: `/v3/projects/bf20b94f6a6c44eca0021766f6d15af5/users/25b1d498383243dab4d93d317ffff149/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.515220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf20b94f6a6c44eca0021766f6d15af5, user_id=25b1d498383243dab4d93d317ffff149, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.518791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.580694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1086/2187] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:07 2022] PUT /identity/v3/projects/bf20b94f6a6c44eca0021766f6d15af5/users/25b1d498383243dab4d93d317ffff149/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:07.587837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.588373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.588904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.616994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:07.618294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:07.695903 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1102/2188] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:07.707321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:07.709683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:07.779897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58239b3c-196d-4e15-a79b-672ed5db8d7d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:07.783866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:07.784564 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:07.784946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:07.785743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:07.797316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:07.798997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:07.812108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1087/2189] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:08.412088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:08.414918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:08.588865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219d0a-d04f-42d0-91db-4e7a8e07e921 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:08.595252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:08.595252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:08.595719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:08.596022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:08.599967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:08.830244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:08.837538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b00633-a37f-4b08-bab9-0cd77557c79e service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:08.858435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1103/2190] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:09.544587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:09.547682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:09.569536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe2c5e-56d7-423e-a64c-099e35ae012c service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:09.576121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:09.577304 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:09.577793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:09.578242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:09.581313 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:09.645360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:09.653644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:09.653644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:09.695320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] MFA Rules not processed for user `25b1d498383243dab4d93d317ffff149`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:09.699406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:09.761109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:09.763828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:09.782691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1088/2191] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:09.798411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=100 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:09.856353 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1104/2192] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:09.861295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:09.886975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad51f9-a630-4909-9ff4-e02fa6d60cb0 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:09.896416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:09.898147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:09.898640 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:09.899092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:09.900389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:09.903464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.066954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1089/2193] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:09 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:10.071218 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:10.095835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77ee3a-e45d-49c2-9a9e-74e7554fb499 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:10.262325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:10.265608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:10.265608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:10.265608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:10.265608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:10.266467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.389502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1105/2194] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:10 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:10.395848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:10.420288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063cbbc-9b0c-4b3a-af30-7d4bdc22f9c1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:10.431592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:10.433144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:10.434006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:10.435522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:10.438673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:10.442393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.514992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1090/2195] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:10.517021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:10.530224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fda6e-041d-4e4d-8a4c-8dd88363bfeb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:10.544449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:10.544449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:10.544449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:10.547452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] PATH_INFO: `/v3/projects/7350fef4c3f746178237f69ccab59bb3/users/8a45161efaad4faa8ee43cb0b8547c1c/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:10.583555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7350fef4c3f746178237f69ccab59bb3, user_id=8a45161efaad4faa8ee43cb0b8547c1c, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:10.590130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.694980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1106/2196] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:10 2022] PUT /identity/v3/projects/7350fef4c3f746178237f69ccab59bb3/users/8a45161efaad4faa8ee43cb0b8547c1c/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:10.700357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:10.736934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2f485-092f-4592-9495-b416459e9a75 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:10.752460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:10.874607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:10.876580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:10.877494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:10.878275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:10.880715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:10.884264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b997-ed3b-434d-a5bf-4b05b38c9702 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.912931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1091/2197] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:10.913323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:10.930381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b9ec4-a693-4507-bfac-9a952592c32d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:10.936261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:10.937277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:10.937562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:10.937844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] PATH_INFO: `/v3/projects/7350fef4c3f746178237f69ccab59bb3/users/8a45161efaad4faa8ee43cb0b8547c1c/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:10.940407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7350fef4c3f746178237f69ccab59bb3, user_id=8a45161efaad4faa8ee43cb0b8547c1c, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:10.955190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:10.984975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1107/2198] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:10 2022] PUT /identity/v3/projects/7350fef4c3f746178237f69ccab59bb3/users/8a45161efaad4faa8ee43cb0b8547c1c/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:12.750617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6120a-9014-4e61-9b9a-6686e58935bc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:12.752144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6120a-9014-4e61-9b9a-6686e58935bc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:12.752631 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6120a-9014-4e61-9b9a-6686e58935bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:12.794510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3eb6120a-9014-4e61-9b9a-6686e58935bc None None] MFA Rules not processed for user `8a45161efaad4faa8ee43cb0b8547c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:12.798715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3eb6120a-9014-4e61-9b9a-6686e58935bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:12.983402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1092/2199] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:12.999000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:13.118352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c84db0-0618-410e-aae0-2febbf2df068 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:13.122583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:13.123261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:13.123551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:13.123838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:13.178260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:13.200861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:13.202846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:13.218713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1108/2200] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:14.926438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:14.927212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:14.927755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:14.961459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] MFA Rules not processed for user `b58b0d520aa1418bb6ad3b5645a826a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:14.963475 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:15.069915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1093/2201] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:16.547372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:16.557719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f63f8-6c00-401b-8af4-63f5ec158ebe service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:16.563190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:16.570304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:16.570304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:16.570304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:16.578818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:16.580806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d187b-9aa2-4649-84e1-12756cf47d44 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:16.594230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1109/2202] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:16 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:18.710538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:18.712351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:18.770556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:18.825322 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3319b-c375-488d-9a73-41916b7a0058 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:18.833176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:18.833414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:18.833827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:18.834272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] PATH_INFO: `/v3/users/77db1d94ede442f68e55ef7b08a9c2d9` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:18.836081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77db1d94ede442f68e55ef7b08a9c2d9)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:18.843319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:19.137235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77db1d94ede442f68e55ef7b08a9c2d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:19.138117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77db1d94ede442f68e55ef7b08a9c2d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:19.156776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77db1d94ede442f68e55ef7b08a9c2d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:19.169251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77db1d94ede442f68e55ef7b08a9c2d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:19.242311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77db1d94ede442f68e55ef7b08a9c2d9'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:19.251469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1094/2203] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:18 2022] DELETE /identity/v3/users/77db1d94ede442f68e55ef7b08a9c2d9 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:19.257003 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:19.257785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:19.257938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:19.278092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:19.279345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:19.357553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1110/2204] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:19.368115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:19.370015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:19.437969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea79c18e-5862-4f7f-885c-06128224ba3f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:19.442019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:19.442693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:19.444617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:19.444994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:19.456526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:19.459061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:19.469940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1095/2205] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:27:19.976498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:19.978189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:19.999484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e59a0-873e-46a3-979f-dece2fb9429c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:20.008559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:20.009799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:20.010214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:20.010715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] PATH_INFO: `/v3/projects/73ac0cda9f374fc6ba89fa8ebacbe603` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:20.013042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ac0cda9f374fc6ba89fa8ebacbe603)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:20.016518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:20.146483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ac0cda9f374fc6ba89fa8ebacbe603', 'request_id': 'req-ae3c8745-dd09-454c-8517-8a96cf5ae083'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.154885 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ac0cda9f374fc6ba89fa8ebacbe603', 'request_id': 'req-ae3c8745-dd09-454c-8517-8a96cf5ae083'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.247992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ac0cda9f374fc6ba89fa8ebacbe603', 'request_id': 'req-ae3c8745-dd09-454c-8517-8a96cf5ae083'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.262546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] The token cache is being invalidate because project 73ac0cda9f374fc6ba89fa8ebacbe603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:20.263332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.263914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.266343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1111/2206] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:19 2022] DELETE /identity/v3/projects/73ac0cda9f374fc6ba89fa8ebacbe603 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:20.271888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:20.284087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510da094-3df5-4aef-9b20-481da32a1395 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:20.357556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:20.369803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:20.369803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:20.369803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:20.369803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] PATH_INFO: `/v3/users/60e65e0617c64761a7929aca27b4f45c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:20.370294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e65e0617c64761a7929aca27b4f45c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:20.370294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:20.574973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e65e0617c64761a7929aca27b4f45c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.576052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e65e0617c64761a7929aca27b4f45c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.604455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e65e0617c64761a7929aca27b4f45c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.614446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e65e0617c64761a7929aca27b4f45c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.683411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e65e0617c64761a7929aca27b4f45c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:20.711822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1096/2207] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:20 2022] DELETE /identity/v3/users/60e65e0617c64761a7929aca27b4f45c => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:21.296045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:21.297795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:21.319777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c8745-dd09-454c-8517-8a96cf5ae083 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:21.383775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:21.391961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:21.392975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:21.394517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:21.394517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] PATH_INFO: `/v3/projects/92dfe44cdafe4286b2cd172f5464812b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:21.394967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92dfe44cdafe4286b2cd172f5464812b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:21.397039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:21.538072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92dfe44cdafe4286b2cd172f5464812b', 'request_id': 'req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.545700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92dfe44cdafe4286b2cd172f5464812b', 'request_id': 'req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.642193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92dfe44cdafe4286b2cd172f5464812b', 'request_id': 'req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.655542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] The token cache is being invalidate because project 92dfe44cdafe4286b2cd172f5464812b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:21.655975 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.656761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.658385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1112/2208] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:21 2022] DELETE /identity/v3/projects/92dfe44cdafe4286b2cd172f5464812b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:21.663428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:21.664918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:21.679427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f020c-bf37-4fae-9a2e-1ec7fc964a30 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:21.743064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:21.743961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:21.744247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:21.744559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] PATH_INFO: `/v3/users/edb0f2660167457a921ecbc82de6bfc6` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:21.745832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb0f2660167457a921ecbc82de6bfc6)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:21.747958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:21.997323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb0f2660167457a921ecbc82de6bfc6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:21.998047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb0f2660167457a921ecbc82de6bfc6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:22.016533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb0f2660167457a921ecbc82de6bfc6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:22.031233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb0f2660167457a921ecbc82de6bfc6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:22.038371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:22.133768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-023f3d6c-df73-46bf-be6e-93ed01a4528c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb0f2660167457a921ecbc82de6bfc6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:22.146508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1097/2209] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:21 2022] DELETE /identity/v3/users/edb0f2660167457a921ecbc82de6bfc6 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:22.904153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:22.907476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:22.940590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcf197-6c28-4ead-9f06-37ed5d0ade48 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:23.090098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:23.091675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:23.092179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:23.093209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] PATH_INFO: `/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:23.095095 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aeaf6fc849b4a29b1bec0994e8add61)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:23.103142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:23.274973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aeaf6fc849b4a29b1bec0994e8add61', 'request_id': 'req-b22ca77a-7362-456e-acca-e06fbf1df861'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:23.283906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aeaf6fc849b4a29b1bec0994e8add61', 'request_id': 'req-b22ca77a-7362-456e-acca-e06fbf1df861'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:23.385357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aeaf6fc849b4a29b1bec0994e8add61', 'request_id': 'req-b22ca77a-7362-456e-acca-e06fbf1df861'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:23.401996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] The token cache is being invalidate because project 4aeaf6fc849b4a29b1bec0994e8add61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:23.402293 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:23.403319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b22ca77a-7362-456e-acca-e06fbf1df861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:23.408698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1113/2210] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:22 2022] DELETE /identity/v3/projects/4aeaf6fc849b4a29b1bec0994e8add61 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:23.421531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:23.422076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:23.422542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:23.502665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:23.508170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:23.615624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1098/2211] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:23.627768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:23.628447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:23.629503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:23.654205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:23.655970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:23.666044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1114/2212] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:23.675417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:23.694703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5f2ff-e4b3-4b5b-a1fd-4563d177ecd9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:23.708565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:23.709844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:23.710420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:23.710797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:23.712162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:23.712711 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:23.716082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:23.740229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1099/2213] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:23.747029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:23.770133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a96b4-22f3-4f55-98ae-f8ba437807e5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:23.776125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:23.777272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:23.778125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:23.778532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:23.781045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:23.784341 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:23.792722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:23.930790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1115/2214] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:23 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:23.935233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:23.952893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca9f9d-f3c4-4e05-ad66-a14ff81ca798 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:24.154371 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:24.155730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:24.156449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:24.156949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:24.158384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:24.162854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:24.289551 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1100/2215] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:23 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:24.294750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:24.330042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dde279-ac00-4ecf-a63b-5f428fdbc2cb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:24.343582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:24.352517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:24.352517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:24.352517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:24.353394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:24.358810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:24.398555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1116/2216] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:24.399395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:24.423960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c44f02-42be-41d3-a1b4-4b123fa22653 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:24.434774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:24.436438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:24.437274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:24.438068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:24.440070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:24.444845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed0b44-fd05-4fd2-9cbb-bb7e281beff2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:24.477427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1101/2217] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:24.479892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:24.500416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005823f-67f6-4767-b922-613c6dca50ab admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:24.507462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:24.508620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:24.509091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:24.509623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] PATH_INFO: `/v3/projects/184c5f95dfcb4cceaf25f6f6e2ae1f20/users/3b0054c665e4437aaf2dd8ab5b85a773/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:24.529752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=184c5f95dfcb4cceaf25f6f6e2ae1f20, user_id=3b0054c665e4437aaf2dd8ab5b85a773, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:24.532863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:24.697270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1117/2218] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:24 2022] PUT /identity/v3/projects/184c5f95dfcb4cceaf25f6f6e2ae1f20/users/3b0054c665e4437aaf2dd8ab5b85a773/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:24.699891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:24.700436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:24.702685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:24.787034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] MFA Rules not processed for user `3b0054c665e4437aaf2dd8ab5b85a773`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:24.791880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:24.907157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:24.943047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1102/2219] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:25.016294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:25.018434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:25.110042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1966480-de47-4b43-a694-a06d5ef6c600 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:25.117598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:25.118783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:25.119323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:25.119850 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:25.136920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:25.140442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fc9b8-d3e5-4488-84b4-a8997516b3db service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:25.158775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1118/2220] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:25.232927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:25.349029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab859bd-45fa-416d-b767-583d8b137c9b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:25.354819 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:25.358941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:25.358941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:25.358941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] PATH_INFO: `/v3/users/3b0054c665e4437aaf2dd8ab5b85a773` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:25.358941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0054c665e4437aaf2dd8ab5b85a773)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:25.367449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:25.641505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0054c665e4437aaf2dd8ab5b85a773'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:25.643064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0054c665e4437aaf2dd8ab5b85a773'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:25.670283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0054c665e4437aaf2dd8ab5b85a773'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:25.683919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0054c665e4437aaf2dd8ab5b85a773'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:25.785964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0054c665e4437aaf2dd8ab5b85a773'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:25.795348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1103/2221] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:25 2022] DELETE /identity/v3/users/3b0054c665e4437aaf2dd8ab5b85a773 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:25.801067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:25.801475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:25.801825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:25.832808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:25.834569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:25.949928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1119/2222] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:25.968210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:25.971259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:26.079139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5494b59-1aa2-4717-9e5f-d14f4e89b2c2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:26.114984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:26.116102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:26.116707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:26.117354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:26.143892 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:26.149858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceb576-fa18-424e-8eb7-91dd433e098e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:26.177493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1104/2223] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:27:26.869053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:26.871773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:26.885137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616225-689f-4dec-af4b-f659c1b2d29e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:26.891160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:26.892113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:26.892431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:26.892940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] PATH_INFO: `/v3/projects/184c5f95dfcb4cceaf25f6f6e2ae1f20` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:26.894497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=184c5f95dfcb4cceaf25f6f6e2ae1f20)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:26.897734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.074311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184c5f95dfcb4cceaf25f6f6e2ae1f20', 'request_id': 'req-b3aea491-d58c-4c1e-a54b-e1c3be949487'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:27.077790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:27.086248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184c5f95dfcb4cceaf25f6f6e2ae1f20', 'request_id': 'req-b3aea491-d58c-4c1e-a54b-e1c3be949487'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:27.155747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184c5f95dfcb4cceaf25f6f6e2ae1f20', 'request_id': 'req-b3aea491-d58c-4c1e-a54b-e1c3be949487'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:27.172085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] The token cache is being invalidate because project 184c5f95dfcb4cceaf25f6f6e2ae1f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:27.172085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:27.172085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-b3aea491-d58c-4c1e-a54b-e1c3be949487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:27.174900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1120/2224] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:26 2022] DELETE /identity/v3/projects/184c5f95dfcb4cceaf25f6f6e2ae1f20 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:27.184167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.185040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.185331 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.220073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:27.221714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:27.234949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:27.309823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1105/2225] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:27.320260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.320773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.321263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.351922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:27.357638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:27.364430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1121/2226] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:27.370019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.382772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae65c1-3f75-4325-8676-4fe2d6d2b73e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.387901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.388646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.388940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.389225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.390044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:27.390365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.392082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.406236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1106/2227] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:27.412454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.425439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94ab5-5f32-4d2f-b764-201822f2b487 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.432232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.438051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.438051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.438051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.438051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.438051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.553492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1122/2228] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:27 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:27.557480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.573639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ba917-b29b-419b-932e-d729324681c6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.655175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.656054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.656579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.657054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.658145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.661503 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.809421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1107/2229] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:27 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:27.814288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.826704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea545d-5f8b-4269-a91e-6e8016b1861e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.833585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.834504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.835168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.835367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.836247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.838249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.850780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1123/2230] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:27.855166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.870508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7769de9-6022-4290-a391-9f8a528895eb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.875745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.876483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.876775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.877062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.877826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.880414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff53754-51ca-45f0-8555-01bcb7724109 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:27.892771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1108/2231] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:27.897816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:27.911031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a576-0907-4222-9912-39be4d3ef2d1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:27.917265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:27.918186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:27.918476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:27.918759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] PATH_INFO: `/v3/projects/c9b591880e9441acac9ddf006a5f9959/users/0e36c7fd357044ccbb59bddd12b61149/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:27.937632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b591880e9441acac9ddf006a5f9959, user_id=0e36c7fd357044ccbb59bddd12b61149, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:27.940704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:28.021173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1124/2232] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:27 2022] PUT /identity/v3/projects/c9b591880e9441acac9ddf006a5f9959/users/0e36c7fd357044ccbb59bddd12b61149/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:28.027188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:28.027485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:28.027778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:28.048780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:28.050060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:28.145530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1109/2233] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:28.156025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:28.159553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:28.257057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8d7c5-5462-4f3a-920f-6dbbbbcbf8fc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:28.261441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:28.262124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:28.262440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:28.262806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:28.274967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:28.276505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5facd-ebb0-43f7-a21d-d2c11cb3b192 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:28.288238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1125/2234] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:33.201785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:33.204817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:33.256293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03889a1a-7512-499f-aa92-3111cadfcbff None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:33.272089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:33.273347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:33.276832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:33.277422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] PATH_INFO: `/v3/users/df40fa149a5e478b9e65c118d1977488` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:33.279075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df40fa149a5e478b9e65c118d1977488)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:33.282488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:33.318412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:33.561794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df40fa149a5e478b9e65c118d1977488'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:33.563800 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df40fa149a5e478b9e65c118d1977488'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:33.596984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df40fa149a5e478b9e65c118d1977488'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:33.608659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df40fa149a5e478b9e65c118d1977488'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:33.720615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df40fa149a5e478b9e65c118d1977488'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:33.732281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1110/2235] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:33 2022] DELETE /identity/v3/users/df40fa149a5e478b9e65c118d1977488 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:33.739234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:33.740085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:33.740251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:33.771679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:33.773933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:33.908829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1126/2236] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:33.924545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:33.927623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:34.010187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155e03-912b-4ef5-8d59-457ee4593118 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:34.016815 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:34.018533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:34.018533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:34.018976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:34.037830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:34.039759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba0b0-95e0-430b-9982-3f7d0bf25930 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:34.053643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1111/2237] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:35.083446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:35.086404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:35.111065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e70a3-ff4f-497f-bdaa-4068dcfc3523 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:35.119185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:35.120585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:35.121021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:35.121623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] PATH_INFO: `/v3/projects/2d88280f17e84da79842ac2e0780fb97` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:35.123517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d88280f17e84da79842ac2e0780fb97)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:35.127225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:35.146614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:35.311394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d88280f17e84da79842ac2e0780fb97', 'request_id': 'req-726aa61c-c848-499f-9e2e-98f2ec6dfa81'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:35.324150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d88280f17e84da79842ac2e0780fb97', 'request_id': 'req-726aa61c-c848-499f-9e2e-98f2ec6dfa81'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:35.391278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d88280f17e84da79842ac2e0780fb97', 'request_id': 'req-726aa61c-c848-499f-9e2e-98f2ec6dfa81'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:35.409766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] The token cache is being invalidate because project 2d88280f17e84da79842ac2e0780fb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:35.410256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:35.412557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-726aa61c-c848-499f-9e2e-98f2ec6dfa81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:35.414522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1127/2238] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:35 2022] DELETE /identity/v3/projects/2d88280f17e84da79842ac2e0780fb97 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:35.420479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:35.421311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:35.421311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:35.447129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:35.453570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:35.582302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1112/2239] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:35.591694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:35.592073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:35.592449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:35.619305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:35.620859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:35.636157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1128/2240] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:35.638613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:35.651655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945846f-d30c-409c-9bee-843d99dd48cf None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:35.666858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:35.678624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:35.680723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:35.684875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:35.685862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:35.686295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:35.688141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:35.770129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1113/2241] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:35.771320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:35.784387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413de9-3830-4a80-920c-097e2789bd1a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:35.807463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:35.808611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:35.808803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:35.809394 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:35.814187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:35.818299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:36.005695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:36.009550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1129/2242] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:35 2022] POST /identity/v3/projects => generated 373 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:36.026676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a2d47f-8ca4-4352-8b39-736e0c6aeb51 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:36.075417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:36.175229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:36.178164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:36.178164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:36.178511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:36.180154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:36.184745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:36.342057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1114/2243] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:36 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:36.347638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:36.377168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b2602-a876-49db-95ff-873233f17c36 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:36.389257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:36.392915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:36.398024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:36.398024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:36.398024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:36.401665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:36.432756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1130/2244] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:36.437249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:36.454116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256bdb7-5d4e-495f-abe3-e540df61984e admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:36.465779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:36.467356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:36.467999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:36.468607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:36.470065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:36.473928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9c285-5a70-48c7-b246-c4914cdad32a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:36.494786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1115/2245] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:36.499205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:36.514731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1017062-b5e3-4fea-86c0-5d0c628a0ce6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:36.524960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:36.526345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:36.526785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:36.527204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] PATH_INFO: `/v3/projects/e838bf49cff94199ac354e9f42d42af2/users/6767e4eeaf094d61a75469e9c7d66eac/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:36.549221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e838bf49cff94199ac354e9f42d42af2, user_id=6767e4eeaf094d61a75469e9c7d66eac, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:36.552214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:36.653137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1131/2246] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:36 2022] PUT /identity/v3/projects/e838bf49cff94199ac354e9f42d42af2/users/6767e4eeaf094d61a75469e9c7d66eac/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:36.658120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368016db-0ba8-47b9-9807-54072a98552e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:36.658426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368016db-0ba8-47b9-9807-54072a98552e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:36.658737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368016db-0ba8-47b9-9807-54072a98552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:36.697492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-368016db-0ba8-47b9-9807-54072a98552e None None] MFA Rules not processed for user `6767e4eeaf094d61a75469e9c7d66eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:36.702333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-368016db-0ba8-47b9-9807-54072a98552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:36.892350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1116/2247] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:36.913664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:36.916292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:37.053725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d021d58-217b-4455-bdd3-1bd337baf0d1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:37.060645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:37.061757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:37.062267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:37.062778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:37.179409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:37.179409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:37.179409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1132/2248] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:27:37.473676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:37.474476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:37.474476 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:37.549449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] MFA Rules not processed for user `0e36c7fd357044ccbb59bddd12b61149`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:37.550065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:37.676111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1117/2249] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:37.682834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:37.686116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:37.755761 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:37.788742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8854e77-343b-43cf-9fe5-4f8339ddf46c service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:37.799741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:37.799741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:37.799741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:37.799741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:37.800289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:37.800289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:37.891450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1133/2250] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:37 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:37.897563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:37.910913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911498c7-fff5-472b-8e05-30b2c5c3b055 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:37.931253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:38.005826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:38.006965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:38.007180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:38.007358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:38.008096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:38.010229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:38.229265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1118/2251] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:37 2022] POST /identity/v3/users => generated 299 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:38.235614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:38.275770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4d7cf-6561-4183-95ee-57404df8c708 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:38.288441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:38.290522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:38.290522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:38.290522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:38.292014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:38.298108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:38.316497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1134/2252] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:38.322612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:38.343205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbadee-865d-4b90-81f0-a3a43627b01d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:38.352179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:38.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:38.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:38.357712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] PATH_INFO: `/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39/users/4b2c7e0398cc41c18b7be1fd7fa185e0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:38.406215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c64e8d93054cec8ddaff71cb6a0b39, user_id=4b2c7e0398cc41c18b7be1fd7fa185e0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:38.412198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:38.580985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1119/2253] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:38 2022] PUT /identity/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39/users/4b2c7e0398cc41c18b7be1fd7fa185e0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:38.586299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:38.603608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1811c02-3162-481c-b627-a44101f029fb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:38.815860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:38.815860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:38.815860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:38.815860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:38.817224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:38.817224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca544a-7f08-4e03-89ea-9605d9a3365b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:38.861451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1135/2254] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:38.868163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:38.914168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440da6b-380e-4dfc-ab3e-12940bd8028a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:38.919883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:38.920888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:38.921177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:38.921464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] PATH_INFO: `/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39/users/4b2c7e0398cc41c18b7be1fd7fa185e0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:38.933173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c64e8d93054cec8ddaff71cb6a0b39, user_id=4b2c7e0398cc41c18b7be1fd7fa185e0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:38.936058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bc78b-6fc9-4fc7-ad9c-622274ff7172 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:38.966310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1120/2255] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:38 2022] PUT /identity/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39/users/4b2c7e0398cc41c18b7be1fd7fa185e0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:39.683258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:39.683925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:39.684488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:39.726297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] MFA Rules not processed for user `6767e4eeaf094d61a75469e9c7d66eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:39.730853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:39.877934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1136/2256] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:42.295158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:42.295698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:42.301642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:42.327617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:42.329071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:42.524709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1121/2257] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:27:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:42.609360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:42.610984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:42.799833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:42.823004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a8354-63be-4873-8983-0ec5eb9c313e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:42.830847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:42.831535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:42.831816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:42.832223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:42.844550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:42.847522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487ca92-9622-4de6-9041-1940665bfe7e service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:42.860924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1137/2258] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:27:43.461181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:43.468082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:43.738793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d91f39-4e21-4547-a2d7-aa0afb8456c7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:43.753010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:43.755689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:43.755689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:43.755943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] PATH_INFO: `/v3/users/c9543123d9554d8e884cf1339ca959a0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:43.759882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9543123d9554d8e884cf1339ca959a0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:43.763434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:43.846422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:44.154204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9543123d9554d8e884cf1339ca959a0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:44.155649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9543123d9554d8e884cf1339ca959a0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:44.200697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9543123d9554d8e884cf1339ca959a0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:44.214369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9543123d9554d8e884cf1339ca959a0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:44.298561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9543123d9554d8e884cf1339ca959a0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:44.318055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1122/2259] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:43 2022] DELETE /identity/v3/users/c9543123d9554d8e884cf1339ca959a0 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:44.322560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:44.323378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:44.323582 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:44.357369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:44.361231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:44.502427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1138/2260] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:44.548877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:44.550630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:44.696165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b2f15-a495-4232-bda0-8b1365dbd5c6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:44.701710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:44.709510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:44.709510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:44.709510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:44.721886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:44.723886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:44.749307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1123/2261] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:27:47.004836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:47.004836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:47.065894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e58482-9702-4cbe-ab4f-746ad3491a76 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:47.097484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:47.100142 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:47.105269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:47.106598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] PATH_INFO: `/v3/projects/f716a9a939ba4ecea613eddc030a65d1` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:47.117822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f716a9a939ba4ecea613eddc030a65d1)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:47.131299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:47.472291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f716a9a939ba4ecea613eddc030a65d1', 'request_id': 'req-729acc7a-424b-43d5-ae99-77e54b352c4b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:47.502927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f716a9a939ba4ecea613eddc030a65d1', 'request_id': 'req-729acc7a-424b-43d5-ae99-77e54b352c4b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:47.603789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f716a9a939ba4ecea613eddc030a65d1', 'request_id': 'req-729acc7a-424b-43d5-ae99-77e54b352c4b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:47.645824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] The token cache is being invalidate because project f716a9a939ba4ecea613eddc030a65d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:47.649092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:47.650473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-729acc7a-424b-43d5-ae99-77e54b352c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:47.664917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1139/2262] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:46 2022] DELETE /identity/v3/projects/f716a9a939ba4ecea613eddc030a65d1 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:47.671436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:47.698614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6b86c-a842-4341-be71-a4a4e9fa8990 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:47.855024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:47.856989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:47.857632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:47.858742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] PATH_INFO: `/v3/users/6ea46ebf71ff474fbf70b4774b684415` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:47.861090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea46ebf71ff474fbf70b4774b684415)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:47.864830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:48.153183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:48.174128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea46ebf71ff474fbf70b4774b684415'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:48.181282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea46ebf71ff474fbf70b4774b684415'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:48.214763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea46ebf71ff474fbf70b4774b684415'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:48.237042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea46ebf71ff474fbf70b4774b684415'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:48.329771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-66501781-6618-48b3-80fd-03261192faa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea46ebf71ff474fbf70b4774b684415'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:48.357606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1124/2263] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:47 2022] DELETE /identity/v3/users/6ea46ebf71ff474fbf70b4774b684415 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:49.099643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:49.100651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:49.101101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:49.102794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1140/2264] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:27:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:27:49.167124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:49.168891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:49.169377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:49.214173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] MFA Rules not processed for user `096d7e0380e84c97a9b72b4203a19abd`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:49.215595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:49.223409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:49.228350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:49.484982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1125/2265] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:27:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:49.654828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94719c2-da10-4cc8-8157-a25159dc273d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:49.679462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:49.681106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:49.681106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:49.681106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] PATH_INFO: `/v3/projects/ebd6db36dfef4e41a6c5032d1df88541` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:49.691352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd6db36dfef4e41a6c5032d1df88541)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:49.703212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:49.749541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:49.753720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:49.764772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:49.830709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adee983-bae9-4239-8edc-372e5ecda975 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:49.851968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:49.858356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:49.858356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:49.858356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:49.891363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:49.893945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbc01b-dc2e-4b7d-9848-a1110df914b0 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:49.926629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1126/2266] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:49.973778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd6db36dfef4e41a6c5032d1df88541', 'request_id': 'req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:49.991843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd6db36dfef4e41a6c5032d1df88541', 'request_id': 'req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:50.112920 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd6db36dfef4e41a6c5032d1df88541', 'request_id': 'req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:50.163140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] The token cache is being invalidate because project ebd6db36dfef4e41a6c5032d1df88541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:50.164021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:50.177904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-59b70e21-dcab-448c-bf8d-192d0fc2b6fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:50.182495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1141/2267] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:49 2022] DELETE /identity/v3/projects/ebd6db36dfef4e41a6c5032d1df88541 => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:50.187294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:50.188682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:50.189213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:50.241038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:50.243552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:50.346471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1127/2268] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:50.348866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:50.349383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:50.349884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:50.427784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:50.430149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:50.498762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:50.509151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1142/2269] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:50.516067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846a97a-de8e-4a67-88d9-12a59a2311d9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:50.535106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:50.537026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:50.537352 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:50.537743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:50.538865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:50.539301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:50.552607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:50.579130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1128/2270] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:50.584488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:50.609282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846b21b-680a-4a34-a66b-aee1bc623627 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:50.649912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:50.651464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:50.652546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:50.653307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:50.654966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:50.668844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:50.910140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1143/2271] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:50 2022] POST /identity/v3/projects => generated 387 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:50.915263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:50.929662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ccfed-2344-44a5-9b39-6eede1de60cf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:51.068083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:51.146166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:51.146166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:51.146166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:51.146166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:51.180484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:51.180484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:51.314946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1129/2272] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:50 2022] POST /identity/v3/users => generated 294 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:27:51.330930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:51.359497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c552-796e-441a-a268-f6ab4089afd3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:51.383733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:51.384484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:51.384782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:51.389417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:51.390325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:51.392025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:51.471531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1144/2273] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:51.476600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:51.504515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e66a2-baa3-4ad6-a1f4-52039703ed2b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:51.515956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:51.516848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:51.517139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:51.517998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:51.522657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:51.524605 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb105b9-cd9f-41d0-ba5c-fee992d06f2b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:51.542562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:51.543125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1130/2274] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:51.554435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acfac2-ca59-4867-a654-71b879516890 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:51.574754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:51.575731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:51.576197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:51.576680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] PATH_INFO: `/v3/projects/38201d157b1740efb60f6691b2f99b3a/users/eea480082c1949aa90a04dd3a2ffa5e3/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:51.617010 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=38201d157b1740efb60f6691b2f99b3a, user_id=eea480082c1949aa90a04dd3a2ffa5e3, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:51.623836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:51.737578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1145/2275] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:51 2022] PUT /identity/v3/projects/38201d157b1740efb60f6691b2f99b3a/users/eea480082c1949aa90a04dd3a2ffa5e3/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:51.751426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:51.751825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:51.752103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:51.831680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] MFA Rules not processed for user `eea480082c1949aa90a04dd3a2ffa5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:51.833456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:51.940243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:51.944097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1131/2276] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:52.083860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b180d6-79a9-4963-9083-1ec3e7b3baaf admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:52.092757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:52.094231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:52.094697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:52.095162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:52.096204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:52.102980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:52.439659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1146/2277] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:51 2022] POST /identity/v3/projects => generated 387 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:52.443060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:52.474488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb656b-b65f-4a16-b4fe-54901b68352f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:52.608108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:52.608885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:52.609190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:52.609632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:52.610568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:52.612664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:52.778348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1132/2278] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:52 2022] POST /identity/v3/users => generated 300 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:52.780230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:52.798172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1a10c-677b-4b2d-894d-730b4bba2dcb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:52.815047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:52.817986 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:52.818286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:52.820889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:52.820889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:52.824811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:52.860832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1147/2279] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:52.861170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:52.868106 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdde9-6c72-44d7-83d0-eeaf63a3494f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:52.876735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:52.877729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:52.878080 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:52.878367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] PATH_INFO: `/v3/projects/c40440eb1cdf451188415da1fadcb373/users/451ec128223e4cbfabd8bff4967c3bdf/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:52.922872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:52.924718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40440eb1cdf451188415da1fadcb373, user_id=451ec128223e4cbfabd8bff4967c3bdf, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:52.925480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:52.929009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:52.949987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01061c4e-4a19-4dd9-98d1-3507c7115741 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:52.974557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:52.978522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:52.979066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:52.979604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] PATH_INFO: `/v3/users/25b1d498383243dab4d93d317ffff149` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:52.981295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b1d498383243dab4d93d317ffff149)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:52.985491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:52.989893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:53.092132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1133/2280] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:52 2022] PUT /identity/v3/projects/c40440eb1cdf451188415da1fadcb373/users/451ec128223e4cbfabd8bff4967c3bdf/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:53.094472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:53.107588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:53.261548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b1d498383243dab4d93d317ffff149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:53.274809 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b1d498383243dab4d93d317ffff149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:53.298773 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b1d498383243dab4d93d317ffff149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:53.305913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b1d498383243dab4d93d317ffff149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:53.305913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9a103ce1-64b3-452a-819f-0764e38652e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b1d498383243dab4d93d317ffff149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:53.329638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8309449-7cff-473e-8fd3-480a55fc3bdb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:53.336934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:53.340200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:53.340200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:53.340200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:53.340200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:53.340661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:53.357459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1134/2281] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:53.362025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:53.364634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:53.385251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70391f77-7721-4a1c-b516-3775d1850e8c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:53.391318 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:53.392670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:53.392974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:53.393345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] PATH_INFO: `/v3/projects/c40440eb1cdf451188415da1fadcb373/users/451ec128223e4cbfabd8bff4967c3bdf/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:53.414151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40440eb1cdf451188415da1fadcb373, user_id=451ec128223e4cbfabd8bff4967c3bdf, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:53.423185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:53.425165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1148/2282] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:52 2022] DELETE /identity/v3/users/25b1d498383243dab4d93d317ffff149 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:53.466595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1135/2283] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:53 2022] PUT /identity/v3/projects/c40440eb1cdf451188415da1fadcb373/users/451ec128223e4cbfabd8bff4967c3bdf/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:53.475872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:53.477492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:53.478026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:53.600618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] MFA Rules not processed for user `451ec128223e4cbfabd8bff4967c3bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:53.602578 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:53.822004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1149/2284] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:53.839330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:53.841718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:53.963186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebea494-899b-406a-98b4-ffcf1a88bbf4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:53.973972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:53.973972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:53.973972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:53.973972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:53.984705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:53.986549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:54.000457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1136/2285] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:54.099709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:54.269412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274add1-5e26-462f-a508-d76c337b2c58 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:54.319820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:54.327006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:54.327525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:54.327915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] PATH_INFO: `/v3/users/eea480082c1949aa90a04dd3a2ffa5e3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:54.334257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea480082c1949aa90a04dd3a2ffa5e3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:54.334257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:54.353729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:54.625046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:54.631317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:54.677632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea480082c1949aa90a04dd3a2ffa5e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:54.683549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60dcab-9f3e-4f84-932d-b01c0819996f service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:54.707338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:54.712787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:54.716230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:54.716230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] PATH_INFO: `/v3/projects/bf20b94f6a6c44eca0021766f6d15af5` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:54.716230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf20b94f6a6c44eca0021766f6d15af5)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:54.716689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea480082c1949aa90a04dd3a2ffa5e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:54.726543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:54.755661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea480082c1949aa90a04dd3a2ffa5e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:54.757063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea480082c1949aa90a04dd3a2ffa5e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:54.761151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-274d91e0-f796-429e-be63-fb3112c45add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea480082c1949aa90a04dd3a2ffa5e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:54.947542 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1150/2286] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:54 2022] DELETE /identity/v3/users/eea480082c1949aa90a04dd3a2ffa5e3 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:54.953862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:54.954214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:54.954427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:55.008130 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf20b94f6a6c44eca0021766f6d15af5', 'request_id': 'req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.029729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf20b94f6a6c44eca0021766f6d15af5', 'request_id': 'req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.038987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:55.043924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:55.138753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf20b94f6a6c44eca0021766f6d15af5', 'request_id': 'req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.177840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] The token cache is being invalidate because project bf20b94f6a6c44eca0021766f6d15af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:55.179709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.182364 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.198191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1137/2287] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:54 2022] DELETE /identity/v3/projects/bf20b94f6a6c44eca0021766f6d15af5 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:55.210040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1151/2288] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:55.216170 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:55.228054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:55.229223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:55.231270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:55.271155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:55.381099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced587e0-4530-4a5c-9a07-2a5bb93e4207 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:55.409929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:55.410658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:55.410951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:55.411236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:55.439509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:55.446665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:55.485468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1152/2289] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:55.491367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ad4b7-9a09-43cc-8d9a-e52c1f9cbb88 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:55.497410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:55.498593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:55.499073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:55.499716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] PATH_INFO: `/v3/users/8a45161efaad4faa8ee43cb0b8547c1c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:55.501320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a45161efaad4faa8ee43cb0b8547c1c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:55.504825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:55.915147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a45161efaad4faa8ee43cb0b8547c1c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.916710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a45161efaad4faa8ee43cb0b8547c1c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.954350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a45161efaad4faa8ee43cb0b8547c1c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:55.968463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a45161efaad4faa8ee43cb0b8547c1c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:56.066545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a45161efaad4faa8ee43cb0b8547c1c'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:56.078784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1138/2290] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:55 2022] DELETE /identity/v3/users/8a45161efaad4faa8ee43cb0b8547c1c => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:27:56.464786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:56.499828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b9e96-a614-46d0-946f-febb0941f43e service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:56.617996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:56.838646 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:56.854677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:56.855186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:56.855677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] PATH_INFO: `/v3/projects/7350fef4c3f746178237f69ccab59bb3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:56.867064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7350fef4c3f746178237f69ccab59bb3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:56.875887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:57.040097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:57.044623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:57.112003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f569c-0cae-4b6b-8e78-ccab1b2ffe84 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:57.120753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7350fef4c3f746178237f69ccab59bb3', 'request_id': 'req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.132576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7350fef4c3f746178237f69ccab59bb3', 'request_id': 'req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.265909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7350fef4c3f746178237f69ccab59bb3', 'request_id': 'req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.291074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] The token cache is being invalidate because project 7350fef4c3f746178237f69ccab59bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:57.291549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.293518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-36be9ae8-41bd-45b8-b6b3-40bfaa4f61b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.295679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1153/2291] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:56 2022] DELETE /identity/v3/projects/7350fef4c3f746178237f69ccab59bb3 => generated 0 bytes in 832 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:57.302892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee48c0-1e64-45bc-b549-c6272ca27232 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.303840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee48c0-1e64-45bc-b549-c6272ca27232 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.304507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee48c0-1e64-45bc-b549-c6272ca27232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.350357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-feee48c0-1e64-45bc-b549-c6272ca27232 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:57.352402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-feee48c0-1e64-45bc-b549-c6272ca27232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:57.388951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:57.390907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.390907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.390907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] PATH_INFO: `/v3/projects/38201d157b1740efb60f6691b2f99b3a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.392196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38201d157b1740efb60f6691b2f99b3a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:57.394217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:57.522610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1154/2292] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:57.525758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.525758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.525758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.550653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:57.553092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:57.560578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:57.603465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38201d157b1740efb60f6691b2f99b3a', 'request_id': 'req-ca9890e3-f08f-47f2-b081-a939a9dbeac5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.618759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38201d157b1740efb60f6691b2f99b3a', 'request_id': 'req-ca9890e3-f08f-47f2-b081-a939a9dbeac5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.672711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:57.695814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1155/2293] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:27:57.697816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:57.706502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38201d157b1740efb60f6691b2f99b3a', 'request_id': 'req-ca9890e3-f08f-47f2-b081-a939a9dbeac5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.710897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4530e-33a0-4f14-a507-f0448a044f86 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:57.716463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:57.717386 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.720846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.722092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.723856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:27:57.724862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:57.726192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:57.728805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] The token cache is being invalidate because project 38201d157b1740efb60f6691b2f99b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:27:57.730448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.731411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:57.735040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1139/2294] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:57 2022] DELETE /identity/v3/projects/38201d157b1740efb60f6691b2f99b3a => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:57.745632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:57.747999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:57.775263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9890e3-f08f-47f2-b081-a939a9dbeac5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:57.788422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:57.790064 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.790659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.791132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] PATH_INFO: `/v3/users/451ec128223e4cbfabd8bff4967c3bdf` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.793377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1156/2295] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:27:57.794926 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451ec128223e4cbfabd8bff4967c3bdf)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:57.802813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:57.811543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:57.830400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:57.854054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fe9dc-5178-477b-91b8-b31ed7a63bef admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:57.875753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:57.876558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:57.882325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:57.882325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:57.882325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:57.882325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75670723-66e5-416f-8f71-ac31ccf0afcc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:58.031605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1157/2296] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:58.034339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:58.034846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:58.036226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:58.118618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451ec128223e4cbfabd8bff4967c3bdf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:58.120782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451ec128223e4cbfabd8bff4967c3bdf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:58.128150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] MFA Rules not processed for user `4b2c7e0398cc41c18b7be1fd7fa185e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:27:58.131522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:58.153629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451ec128223e4cbfabd8bff4967c3bdf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:58.172258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451ec128223e4cbfabd8bff4967c3bdf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:58.266949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451ec128223e4cbfabd8bff4967c3bdf'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:27:58.283370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1158/2297] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:27:58.290320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:58.292259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:58.304087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1140/2298] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:27:57 2022] DELETE /identity/v3/users/451ec128223e4cbfabd8bff4967c3bdf => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:58.321294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:58.327950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:58.489838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c119c7-f8c0-45e2-8202-eb59e945d1c4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:58.612955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4554d-dec1-4c9e-b6e0-6a3b71bf7509 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:58.636634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:58.638104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:58.640455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:58.640992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:58.651265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:58.700225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:58.714474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:58.716652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:58.718613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:58.719878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:58.722356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:58.727331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:58.783161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:27:58.813806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:58.820164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:58.846798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1141/2299] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:27:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:27:58.880613 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1159/2300] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:27:58 2022] POST /identity/v3/users => generated 292 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:27:58.881490 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:58.897880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3970737-44f1-45b4-af9f-332ccf51ffd8 service glance] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:58.915113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:58.916222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:58.916768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:58.917268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:58.919436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:58.924611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:58.955323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1142/2301] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:27:58.960150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:58.979764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463bf9-278e-4447-b27c-abfa51324923 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:59.002401 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:59.006587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:59.009118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:59.012393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:59.015086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:59.023928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:59.073221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1160/2302] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:27:59.079964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:59.109912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9137-9637-4321-ad07-1ea093c61dd6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:27:59.118823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:27:59.121682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:59.122097 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:59.124044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] PATH_INFO: `/v3/projects/dcbf636efa8340ca8e113a01b6218519/users/98f100dc7f5e4baf99cb6646207aa5d2/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:59.226521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcbf636efa8340ca8e113a01b6218519, user_id=98f100dc7f5e4baf99cb6646207aa5d2, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:27:59.229570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a9511-4963-4c80-a1b6-3f97aea53796 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:27:59.931025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:27:59.936563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:27:59.957959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1143/2303] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:27:59 2022] PUT /identity/v3/projects/dcbf636efa8340ca8e113a01b6218519/users/98f100dc7f5e4baf99cb6646207aa5d2/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:27:59.959268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7297-15d7-4fad-b30a-7396c6aabe55 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:27:59.964464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7297-15d7-4fad-b30a-7396c6aabe55 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:27:59.964976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c7297-15d7-4fad-b30a-7396c6aabe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:27:59.971628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cade4b-6da5-4259-96cd-d9ce4798d13a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:00.123140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b43c7297-15d7-4fad-b30a-7396c6aabe55 None None] MFA Rules not processed for user `98f100dc7f5e4baf99cb6646207aa5d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:00.126922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b43c7297-15d7-4fad-b30a-7396c6aabe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:00.151342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:00.153633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:00.153999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:00.154350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] PATH_INFO: `/v3/projects/c40440eb1cdf451188415da1fadcb373` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:00.156724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40440eb1cdf451188415da1fadcb373)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:00.159454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:00.522914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1144/2304] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:27:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:00.524801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:00.525284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:00.525739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:00.566562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:00.569292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:00.581777 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40440eb1cdf451188415da1fadcb373', 'request_id': 'req-a78ae615-d4fb-455c-b121-0894c272563b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:00.598121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40440eb1cdf451188415da1fadcb373', 'request_id': 'req-a78ae615-d4fb-455c-b121-0894c272563b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:00.706034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40440eb1cdf451188415da1fadcb373', 'request_id': 'req-a78ae615-d4fb-455c-b121-0894c272563b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:00.739446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] The token cache is being invalidate because project c40440eb1cdf451188415da1fadcb373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:00.739740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:00.740117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1145/2305] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:28:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:00.741094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:00.748514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:00.772354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1161/2306] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:27:59 2022] DELETE /identity/v3/projects/c40440eb1cdf451188415da1fadcb373 => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:00.773274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:00.777991 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:00.929707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:00.963666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ae615-d4fb-455c-b121-0894c272563b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:00.970656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a05a6-22e9-42b6-85d9-bfb87e6926c7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:00.984905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:00.985594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:00.985878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:00.986163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:00.988297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:00.989836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:00.990735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:00.991204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:00.991675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:00.993557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:00.995845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.003458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:01.015025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.015025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba769a2-e87c-4bd0-a1fe-08dacd567e1d service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.042753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1162/2307] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:28:01.044159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7ae2-6c74-4779-8b7c-ea43d8ef7dec None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.044477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7ae2-6c74-4779-8b7c-ea43d8ef7dec None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.044840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a7ae2-6c74-4779-8b7c-ea43d8ef7dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.100670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-cc2a7ae2-6c74-4779-8b7c-ea43d8ef7dec None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:01.102318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cc2a7ae2-6c74-4779-8b7c-ea43d8ef7dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:01.103040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1146/2308] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:00 2022] POST /identity/v3/projects => generated 385 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:01.108033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.112728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:01.336584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1163/2309] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:01.338181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.338466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.338747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.377662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd9e23c-e80c-4469-9b1a-760b42971e2a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:01.390012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:01.391333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:01.395882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:01.403386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.403386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.403386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.405093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.413783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.413992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1164/2310] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:01.419067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.447878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1cc7d-ecba-4c7c-8ee5-83dde091f047 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:01.467044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:01.467745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.469398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.469703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.474545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:01.476665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.479993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.519408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1165/2311] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:28:01.522311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.543338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f86c2-8000-474d-b3f6-335f58891acc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:01.558314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:01.564829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.564829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.564829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.564829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.565340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1147/2312] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:01 2022] POST /identity/v3/users => generated 299 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:28:01.565432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.567446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.597029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc5c59-39e0-4f5a-8ddb-410d3c7c091a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:01.613242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:01.614905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.615428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.616218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.617276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.619654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.649323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1148/2313] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:01.651672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.686272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c1942-5385-4897-b60c-4292fd174b63 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:01.700365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:01.701246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:01.701515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:01.701802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] PATH_INFO: `/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc/users/d3aca8301d0a4d8b84a8d441b4d49bef/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:01.725976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1166/2314] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:01 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:01.727107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:01.744528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64de7c08e0a24f7ca6f635eea9234bcc, user_id=d3aca8301d0a4d8b84a8d441b4d49bef, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:01.749113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:01.883303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1149/2315] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:01 2022] PUT /identity/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc/users/d3aca8301d0a4d8b84a8d441b4d49bef/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:01.888193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.013083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868b758-ea83-4729-9382-27458a960f51 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.152439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068af238-71f7-4874-b3d8-98b3e8002394 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.176246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.177547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.177860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.178499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.178850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:02.183897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:02.237116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1150/2316] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:02.238135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.271166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc200d9-70c3-4be2-8bd0-e8addd35f6fd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.282866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.283790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.284081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.284384 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] PATH_INFO: `/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc/users/d3aca8301d0a4d8b84a8d441b4d49bef/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.291134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=64de7c08e0a24f7ca6f635eea9234bcc, user_id=d3aca8301d0a4d8b84a8d441b4d49bef, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:02.293914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda36400-b9b4-48c1-bb0d-48f1ea3ba1ae admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:02.350281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1151/2317] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:02 2022] PUT /identity/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc/users/d3aca8301d0a4d8b84a8d441b4d49bef/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:02.370361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.370726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.370918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.396097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.399189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.399491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.399781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.400598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:02.402681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:02.423998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:02.433702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:02.461518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] MFA Rules not processed for user `d3aca8301d0a4d8b84a8d441b4d49bef`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:02.465340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:02.533769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1167/2318] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:01 2022] POST /identity/v3/users => generated 289 bytes in 807 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:02.540518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.566268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd67c5-0ad8-4b6b-adef-a76a58489335 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.673993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1152/2319] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:02.685038 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.686495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:02.792464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.793186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.793480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.793765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.794710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:02.807092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:02.845644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1168/2320] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:02.846901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.882465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c22e6-c453-48b1-87c8-3eb417726ad1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.908756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.908756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.908756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.912074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.914547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:02.922042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:02.963016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb0a6e-25e8-4268-899d-3f4c775e28e0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:02.968904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1169/2321] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:02.970747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:02.977437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:02.983050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:02.983427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:02.983859 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:02.991114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:02.998964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2fa41-a818-45f7-bb20-12e773b42ce3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:03.024403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:03.026336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:03.028192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:03.028192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] PATH_INFO: `/v3/projects/20afd888f9674005a9264680e5292766/users/a141dbc75633472386336220f6412b74/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:03.088475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20afd888f9674005a9264680e5292766, user_id=a141dbc75633472386336220f6412b74, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:03.092972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81af10-a6ea-464b-9f9d-b8253db3241f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:03.176981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:03.178776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:03.188557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1170/2322] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:02 2022] PUT /identity/v3/projects/20afd888f9674005a9264680e5292766/users/a141dbc75633472386336220f6412b74/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:03.201589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:03.201890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:03.202177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:03.225051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1153/2323] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:03.260475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] MFA Rules not processed for user `a141dbc75633472386336220f6412b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:03.261828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:03.447735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1171/2324] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:03.462750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:03.481963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2118331-231a-4fe9-b28b-54d9e9ba3c05 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:03.623743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:03.638740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:03.638740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:03.638740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:03.639198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:03.685603 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:03.695301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dec6c-8c98-49b9-8b95-aaa0db266973 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:03.784135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1154/2325] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:12.717228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:12.728393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b925df0-02b4-49da-8698-fa4a8f33b311 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:12.732295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:12.733260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:12.733541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:12.733821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:12.811170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:12.858648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:12.860769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:12.881319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1172/2326] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:13.751298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:16.789125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:16.790186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:16.790186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:16.818901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] MFA Rules not processed for user `0e36c7fd357044ccbb59bddd12b61149`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:16.819084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:16.891893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1155/2327] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:18.843090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:18.845536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:18.964854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:18.967048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:18.992616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87300854-bf8d-4b3f-9e17-cd492155770e service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:18.999812 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:19.000772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:19.001730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:19.002220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:19.005499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:19.025448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:19.027131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4a4bc-f520-400e-a2de-3a43ca493355 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:19.043688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1173/2328] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:19.097809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb00e58-a8ae-40b8-8b1f-f819bdc783c5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:19.105353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:19.106179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:19.106523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:19.106966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:19.130017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:19.135210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:19.159569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1156/2329] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:20.577390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:20.579560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:20.579560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:20.621485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:20.621485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:20.720664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1174/2330] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:28:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:20.741724 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:20.921842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8f16e-7310-4b6c-8c1a-4fe6f515bd7b service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:20.934333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:20.936255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:20.937045 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:20.937777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:20.955466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:20.957943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c903802-a6d1-4ddb-9c6b-0a2d5e898799 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:20.975753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1157/2331] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:28:21.666557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:21.848629 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad9a8f-c88a-4934-be32-8a4b8a06dc7a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:21.858770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:21.862474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:21.863018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:21.865383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] PATH_INFO: `/v3/users/98f100dc7f5e4baf99cb6646207aa5d2` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:21.868234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f100dc7f5e4baf99cb6646207aa5d2)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:21.873788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:21.907672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:22.142320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f100dc7f5e4baf99cb6646207aa5d2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:22.161800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f100dc7f5e4baf99cb6646207aa5d2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:22.193511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f100dc7f5e4baf99cb6646207aa5d2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:22.194957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f100dc7f5e4baf99cb6646207aa5d2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:22.196087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f100dc7f5e4baf99cb6646207aa5d2'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:22.219216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15769aba-ac73-4a5c-b9a2-2f0ac995eecc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.220814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15769aba-ac73-4a5c-b9a2-2f0ac995eecc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.221448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15769aba-ac73-4a5c-b9a2-2f0ac995eecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.266555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-15769aba-ac73-4a5c-b9a2-2f0ac995eecc None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:22.271830 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15769aba-ac73-4a5c-b9a2-2f0ac995eecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.294638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1158/2332] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:22.302443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1175/2333] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:21 2022] DELETE /identity/v3/users/98f100dc7f5e4baf99cb6646207aa5d2 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:22.309293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead74d4f-03dc-4590-82e4-3da9ffefce9f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.309648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead74d4f-03dc-4590-82e4-3da9ffefce9f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.310105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead74d4f-03dc-4590-82e4-3da9ffefce9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.320487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:22.328634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.371529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ead74d4f-03dc-4590-82e4-3da9ffefce9f None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:22.373158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ead74d4f-03dc-4590-82e4-3da9ffefce9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.399128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:22.501942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d4d6b-fc5d-4549-9a7f-b848c9d99856 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:22.505624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1159/2334] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:22.508126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.508844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.509876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.512977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1160/2335] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:28:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:28:22.515930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:22.517908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:22.518799 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.519204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.519383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.520923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.647619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:22.649474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386deb-7930-433e-93f1-b4a6a92bb60c service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:22.655126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb403dff-0e42-4440-b28f-a52e7992f8f9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:22.665690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1176/2336] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:28:22.665690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.665690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.665690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.667399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:22.668389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.668762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.669112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.680773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:22.682687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced90f93-13c6-469e-9dbd-636498002202 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:22.687462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:22.693136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.696785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1161/2337] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:22.698716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.699256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.700037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.729965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:22.731274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:22.735214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:22.759597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1162/2338] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:22.776595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:22.795253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b09387-aace-47c4-aed9-2154cb75307d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:22.809006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:22.810753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:22.811409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:22.811904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:22.833087 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:22.839990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:22.847831 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1163/2339] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:28:22.860989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1177/2340] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:28:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:23.692373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:23.695677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:23.741565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800cee3c-7fdb-46a1-ad3a-fa0d96a0b320 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:23.748407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:23.750543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:23.751141 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:23.751597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] PATH_INFO: `/v3/projects/dcbf636efa8340ca8e113a01b6218519` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:23.753434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcbf636efa8340ca8e113a01b6218519)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:23.754091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:23.757262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:23.762216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:23.880380 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b101a-fe48-4cb1-bc83-d86751c1f1ef None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:23.887069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:23.888432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:23.889139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:23.889983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:23.907989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:23.913390 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:23.968562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1178/2341] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:23.997511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcbf636efa8340ca8e113a01b6218519', 'request_id': 'req-88b82749-ae0b-4761-a35a-4abeffb55cd6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.014911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcbf636efa8340ca8e113a01b6218519', 'request_id': 'req-88b82749-ae0b-4761-a35a-4abeffb55cd6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.096595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcbf636efa8340ca8e113a01b6218519', 'request_id': 'req-88b82749-ae0b-4761-a35a-4abeffb55cd6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.124854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] The token cache is being invalidate because project dcbf636efa8340ca8e113a01b6218519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:24.125684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.126857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-88b82749-ae0b-4761-a35a-4abeffb55cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.132853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1164/2342] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:23 2022] DELETE /identity/v3/projects/dcbf636efa8340ca8e113a01b6218519 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:24.141426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:24.164284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b671d-43a7-438e-833b-8415e4075f08 service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:24.260962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:24.262108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:24.262593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:24.263043 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] PATH_INFO: `/v3/users/d3aca8301d0a4d8b84a8d441b4d49bef` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:24.264719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3aca8301d0a4d8b84a8d441b4d49bef)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:24.267022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:24.527729 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc70a7-0e87-4cb9-aa5c-d7dde269d236 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:24.528272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc70a7-0e87-4cb9-aa5c-d7dde269d236 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:24.528895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc70a7-0e87-4cb9-aa5c-d7dde269d236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:24.559485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-15fc70a7-0e87-4cb9-aa5c-d7dde269d236 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:24.580537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15fc70a7-0e87-4cb9-aa5c-d7dde269d236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:24.588800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3aca8301d0a4d8b84a8d441b4d49bef'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.621038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3aca8301d0a4d8b84a8d441b4d49bef'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.622895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:24.652664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:24.670518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3aca8301d0a4d8b84a8d441b4d49bef'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.671682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3aca8301d0a4d8b84a8d441b4d49bef'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.674071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2f6c2e1e-40c1-48a5-9928-8d2578706063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3aca8301d0a4d8b84a8d441b4d49bef'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:24.716858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1165/2343] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:24.728715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:24.730687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:24.732723 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:24.748869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1166/2344] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:28:24 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:28:24.750382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:24.771779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:24.778601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1179/2345] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:24 2022] DELETE /identity/v3/users/d3aca8301d0a4d8b84a8d441b4d49bef => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:24.784718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2bdcd-fcbd-452a-9656-f6819a6c67a4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:24.786603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2bdcd-fcbd-452a-9656-f6819a6c67a4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:24.787123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2bdcd-fcbd-452a-9656-f6819a6c67a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:24.848178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2eb2bdcd-fcbd-452a-9656-f6819a6c67a4 None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:24.852037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2eb2bdcd-fcbd-452a-9656-f6819a6c67a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:24.934028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec9a75-3a78-4f7e-ad93-f61b3a03f253 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:24.942970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:24.943647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:24.944136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:24.944796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.020693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:25.022903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:25.033347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1167/2346] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:28:25.089641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1180/2347] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:28:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:25.141213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:25.143529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:25.193026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0242157-33b2-49ad-adcc-74ed0bfca70f service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:25.210260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:25.211510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:25.212061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:25.212652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.245136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:25.248669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557816e-2f05-468d-ad9a-b8012dff92b0 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:25.287032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1168/2348] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:25.617832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:25.618571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:25.622285 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.623123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1181/2349] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:28:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 18:28:25.631059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:25.631572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:25.632076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.679619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:25.682236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:25.803862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:25.836586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:25.839598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:25.857054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1169/2350] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:28:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:25.886482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:25.914006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d1939-deb0-44c0-bd95-84967772e49f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:25.921072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:25.922242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:25.923732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:25.924670 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.940334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:25.942437 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f222ec-5996-4ce6-9e67-f5a7810fffdc service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:25.954873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1170/2351] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:25.956409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680a1bb-6078-4cf8-9504-7e4bf2464b2f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:25.974038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:25.974038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:25.974038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:25.974038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] PATH_INFO: `/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:25.974546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64de7c08e0a24f7ca6f635eea9234bcc)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:25.975238 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:26.084558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.085151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.085750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.120586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:26.125937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:26.138785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64de7c08e0a24f7ca6f635eea9234bcc', 'request_id': 'req-72fd1002-d26a-4e69-9e27-c18048c840d4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:26.151538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64de7c08e0a24f7ca6f635eea9234bcc', 'request_id': 'req-72fd1002-d26a-4e69-9e27-c18048c840d4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:26.227795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1171/2352] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:26.243778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:26.259589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64de7c08e0a24f7ca6f635eea9234bcc', 'request_id': 'req-72fd1002-d26a-4e69-9e27-c18048c840d4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:26.274236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] The token cache is being invalidate because project 64de7c08e0a24f7ca6f635eea9234bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:26.274557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:26.275330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-72fd1002-d26a-4e69-9e27-c18048c840d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:26.277052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1182/2353] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:25 2022] DELETE /identity/v3/projects/64de7c08e0a24f7ca6f635eea9234bcc => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:26.283323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.283630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.284088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.313377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:26.314566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:26.338783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beb0ba-94be-480c-adaa-14effd035e48 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:26.341266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:26.345728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:26.346533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.346906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.347303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.349962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:26.369161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:26.371728 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294ec8e-35dd-4888-8369-8c620ab51c83 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:26.386931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1172/2354] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:28:26.404434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1183/2355] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:26.418172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.418599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.419075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.446521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:26.448832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:26.462828 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1173/2356] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:26.467923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:26.480180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ed121-412c-4842-b46b-2bac3ae6bb45 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:26.487073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:26.488017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.488744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.489199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.490259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:26.490757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:26.492882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:26.517343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1184/2357] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:28:26.523628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:26.544863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d626b-014c-4f74-8da5-c1e6b9b69c51 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:26.556989 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:26.558264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.558760 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.559282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.561078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:26.564866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:26.721468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1174/2358] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:26 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:28:26.725108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:26.762360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cd0e1-5688-4337-9c2f-7d65c0db654e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:26.898940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:26.899760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:26.900247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:26.900775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:26.901793 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:26.904047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:27.047954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1185/2359] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:26 2022] POST /identity/v3/users => generated 292 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:27.057870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:27.080375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18186e6c-9ccc-4b0f-9e85-1f4c6f8a95d5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:27.091688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:27.092894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:27.093432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:27.093922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:27.095376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:27.099198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:27.123166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1175/2360] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:27.127415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:27.145441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72122949-b2a9-45d2-b421-82bdc35e84eb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:27.151537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:27.152339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:27.152741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:27.153172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:27.153941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:27.155837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fcd3a-7e97-41c4-b04c-089e0cadbd3e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:27.169923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1186/2361] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:27.178193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:27.214591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce1059-7b2f-400e-860f-fdb279a4f176 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:27.222954 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:27.231301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:27.231301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:27.231301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] PATH_INFO: `/v3/projects/48d14f5398f04c9b89a909d5e356b70e/users/90ea0b068ffd4619b33d940a4d6c672a/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:27.274983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d14f5398f04c9b89a909d5e356b70e, user_id=90ea0b068ffd4619b33d940a4d6c672a, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:27.281741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:27.383979 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1176/2362] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:27 2022] PUT /identity/v3/projects/48d14f5398f04c9b89a909d5e356b70e/users/90ea0b068ffd4619b33d940a4d6c672a/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:27.413226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:27.413706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:27.414157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:27.511782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] MFA Rules not processed for user `90ea0b068ffd4619b33d940a4d6c672a`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:27.513105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:27.708922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1187/2363] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:27.724499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:27.726813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:27.746278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:27.854874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77031a-7701-4510-9845-70fba3969c91 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:27.869779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:27.871426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:27.872192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:27.875944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:27.913291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:27.914794 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ddf9e0-1ab6-42ff-b715-45ba0f9f6c1d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:27.941342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1177/2364] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:31.920533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:31.925257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:31.992922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:32.158074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca3dd7-c794-4f81-a5ff-f677e9d03040 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:32.162974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:32.167014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:32.167399 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:32.167779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] PATH_INFO: `/v3/users/6767e4eeaf094d61a75469e9c7d66eac` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:32.169990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6767e4eeaf094d61a75469e9c7d66eac)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:32.173417 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:32.515495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6767e4eeaf094d61a75469e9c7d66eac'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:32.528255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6767e4eeaf094d61a75469e9c7d66eac'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:32.548342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6767e4eeaf094d61a75469e9c7d66eac'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:32.549505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6767e4eeaf094d61a75469e9c7d66eac'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:32.551012 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6767e4eeaf094d61a75469e9c7d66eac'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:32.715440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1188/2365] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:31 2022] DELETE /identity/v3/users/6767e4eeaf094d61a75469e9c7d66eac => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:32.722674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:32.724011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:32.724378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:32.775205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:32.779769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:32.885387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1178/2366] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:32.900145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:32.901181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:33.019024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe82864-f910-4271-b871-79f066ba7813 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:33.028787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:33.029781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:33.030207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:33.030616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:33.043673 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:33.046433 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13475cbe-0e44-41e5-8f04-65fba1e7c88d service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:33.084152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1189/2367] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:33.397365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:33.400804 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:33.424737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e451a-035f-4fc1-8a86-ba87b33fed13 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:33.432350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:33.433274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:33.433553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:33.433840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] PATH_INFO: `/v3/projects/e838bf49cff94199ac354e9f42d42af2` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:33.435229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e838bf49cff94199ac354e9f42d42af2)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:33.437650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:33.581096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e838bf49cff94199ac354e9f42d42af2', 'request_id': 'req-ba0c0f18-02ab-451a-abbb-f682e57a6746'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:33.595996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e838bf49cff94199ac354e9f42d42af2', 'request_id': 'req-ba0c0f18-02ab-451a-abbb-f682e57a6746'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:33.689982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e838bf49cff94199ac354e9f42d42af2', 'request_id': 'req-ba0c0f18-02ab-451a-abbb-f682e57a6746'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:33.731372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] The token cache is being invalidate because project e838bf49cff94199ac354e9f42d42af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:33.731372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:33.737175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-ba0c0f18-02ab-451a-abbb-f682e57a6746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:33.747102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1179/2368] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:33 2022] DELETE /identity/v3/projects/e838bf49cff94199ac354e9f42d42af2 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:33.754735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:33.755116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:33.755701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:33.808412 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:33.810821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:33.898533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:33.962771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1190/2369] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:33.975823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:33.975823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:33.975823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.005164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:34.007509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:34.024587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1180/2370] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:34.024860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.037376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67108e84-04ba-44fe-8a08-74d13e0bec63 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.044117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.044884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.045169 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.045448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.046200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:34.046510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.048182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.063270 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1191/2371] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:28:34.070010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.081542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac505b2-dd9d-4ffc-84cd-5208389800c9 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.087571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.088265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.088597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.088886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.089715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.091776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.177852 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1181/2372] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:34 2022] POST /identity/v3/projects => generated 397 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:34.182611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.193615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30da00-e958-43a2-b5d1-ac4174f1e39d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.276589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.277300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.277604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.277889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.278611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.280832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.388343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1192/2373] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:34 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:34.392974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.405480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:34.413583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82cf67-4175-42eb-8323-7a3994df0d0b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.421180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.422129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.422645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.423159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.424557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.428297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.447148 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1182/2374] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:34.451266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.466232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe0ccd-7e2d-4061-9842-5fa082490ef3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.471338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.472140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.472526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.472849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.474337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.476240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f21949-7bfe-471c-95cf-be1731abe7ad admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.488196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1193/2375] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:34.493241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.503987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43fa86-d7e2-43b6-9194-3eed7ce8b646 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.509255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.510260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.510780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.511389 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] PATH_INFO: `/v3/projects/f36dd5b66ec6412c90f3d5384285feeb/users/9e9baf3d271242db85e7cad270d8b3c5/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.529798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36dd5b66ec6412c90f3d5384285feeb, user_id=9e9baf3d271242db85e7cad270d8b3c5, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.532619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.605053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1183/2376] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:34 2022] PUT /identity/v3/projects/f36dd5b66ec6412c90f3d5384285feeb/users/9e9baf3d271242db85e7cad270d8b3c5/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:34.617255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.617565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.617857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.649086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] MFA Rules not processed for user `9e9baf3d271242db85e7cad270d8b3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:34.650477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:34.724862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1194/2377] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:34.736570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:34.738482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:34.798635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fb215-d36c-4e5e-8b77-17fb45b2a352 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:34.802920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:34.803590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:34.803876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:34.804160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:34.813312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:34.814773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77511b-5826-4570-a999-d272e84bd67d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:34.827743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1184/2378] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:34 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:42.575174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:42.577652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:42.638603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:42.686080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469c0a7-06df-4b8a-8627-8d889f95a1ff None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:42.697616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:42.701170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:42.701703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:42.702215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] PATH_INFO: `/v3/users/a141dbc75633472386336220f6412b74` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:42.704953 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a141dbc75633472386336220f6412b74)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:42.708455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:42.977476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a141dbc75633472386336220f6412b74'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:42.995107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a141dbc75633472386336220f6412b74'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.024024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a141dbc75633472386336220f6412b74'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.027254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a141dbc75633472386336220f6412b74'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.029247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a8d5be14-13ce-48aa-990f-f62a9c0677b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a141dbc75633472386336220f6412b74'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.091720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:43.092441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:43.092441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:43.113589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:43.115554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:43.127387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1195/2379] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:42 2022] DELETE /identity/v3/users/a141dbc75633472386336220f6412b74 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:43.133155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:43.133463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:43.133713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:43.157775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:43.162958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:43.257727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1185/2380] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:28:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:43.284433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:43.287459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:43.293661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1196/2381] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:43.303771 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:43.305286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:43.399838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51724a4-b7d2-42a6-882c-f9f9aab79e22 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:43.406278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:43.407707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:43.408126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:43.408598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:43.425846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:43.427926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:43.436999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:43.460197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061a0b-1d04-4a39-bae8-855adb1813c4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:43.472633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1197/2382] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:43.502358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:43.540119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:43.542443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:43.589842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af217c-9abb-45dd-ae16-2dca6f5de361 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:43.612606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:43.612606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:43.612606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:43.612606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] PATH_INFO: `/v3/users/90ea0b068ffd4619b33d940a4d6c672a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:43.612934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ea0b068ffd4619b33d940a4d6c672a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:43.616956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:43.631068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:43.632368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:43.632898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:43.633960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:43.737022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:43.739576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67270620-5048-4f15-86bb-1a3870a90fc4 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:43.751065 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1186/2383] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:28:43.884782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ea0b068ffd4619b33d940a4d6c672a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.909937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ea0b068ffd4619b33d940a4d6c672a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.937230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ea0b068ffd4619b33d940a4d6c672a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.938079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ea0b068ffd4619b33d940a4d6c672a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:43.938884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ea0b068ffd4619b33d940a4d6c672a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.062774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1198/2384] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:43 2022] DELETE /identity/v3/users/90ea0b068ffd4619b33d940a4d6c672a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:44.070797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:44.071090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:44.071402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:44.114505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:44.135622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:44.366156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1187/2385] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:44.377861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:44.379638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:44.498573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:44.501338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:44.515164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959d2c7-a15d-4eb0-bebc-6c8d113981c1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:44.521675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:44.522743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:44.522876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:44.523222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] PATH_INFO: `/v3/projects/20afd888f9674005a9264680e5292766` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:44.524618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20afd888f9674005a9264680e5292766)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:44.527702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:44.531688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6a44b-ab59-405a-889b-8aa4137ac089 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:44.537415 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:44.538103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:44.538389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:44.538669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:44.559344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:44.561373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:44.578487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1199/2386] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:44.665039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:44.677755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:44.769754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279658b-d716-4f77-9a37-83f44225ae6c service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:44.802279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20afd888f9674005a9264680e5292766', 'request_id': 'req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.837068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20afd888f9674005a9264680e5292766', 'request_id': 'req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.861120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:44.931167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:44.932105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:44.932432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:44.932712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] PATH_INFO: `/v3/users/9e9baf3d271242db85e7cad270d8b3c5` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:44.934040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9baf3d271242db85e7cad270d8b3c5)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:44.940670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:44.961689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20afd888f9674005a9264680e5292766', 'request_id': 'req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.979066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] The token cache is being invalidate because project 20afd888f9674005a9264680e5292766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:44.979471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.980498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-61c21100-38b1-441f-b2bc-b68d8fb2a9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:44.982589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1188/2387] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:44 2022] DELETE /identity/v3/projects/20afd888f9674005a9264680e5292766 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:44.989461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146a94-f894-441f-b7f2-7451f8891dbb None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:44.989887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146a94-f894-441f-b7f2-7451f8891dbb None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:44.990309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9146a94-f894-441f-b7f2-7451f8891dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.000923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:45.056361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b9146a94-f894-441f-b7f2-7451f8891dbb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:45.056361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b9146a94-f894-441f-b7f2-7451f8891dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:45.077191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1189/2388] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:45.089145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.090439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.091440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.185553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:45.187532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:45.203418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1190/2389] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:45.209074 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:45.238387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3c9f-c1b2-4778-b143-7596b3b6157b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:45.250745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:45.254029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.254029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.254029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.255620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:45.256488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:45.262046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:45.287695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9baf3d271242db85e7cad270d8b3c5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:45.297283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1191/2390] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:28:45.317961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:45.320114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9baf3d271242db85e7cad270d8b3c5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:45.343831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9baf3d271242db85e7cad270d8b3c5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:45.344976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9baf3d271242db85e7cad270d8b3c5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:45.346023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c336ba7-2fe0-4adb-b668-4f75257bf9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9baf3d271242db85e7cad270d8b3c5'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:45.348652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eaf6bb-193b-4216-9d7a-60340e23edaf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:45.366858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:45.370240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.373985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.376223 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.378844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:45.383659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:45.491283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1200/2391] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:44 2022] DELETE /identity/v3/users/9e9baf3d271242db85e7cad270d8b3c5 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:45.498914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.500902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.501538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.521738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1192/2392] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:45 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:45.523430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:45.526316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:45.577760 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:45.580197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:45.747517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1201/2393] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:45.752890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:45.759841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:45.777533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cdf16-e1b3-42a9-991c-2e29ee70206d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:45.840509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39545cf-d3cf-4cf4-b51f-4073d8755323 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:45.845584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:45.848893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.859202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:45.860268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.860683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:45.860867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] PATH_INFO: `/v3/projects/48d14f5398f04c9b89a909d5e356b70e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.861221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:45.861946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:45.863090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:45.868447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:45.876904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d14f5398f04c9b89a909d5e356b70e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:45.878887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:45.997112 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1193/2394] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:45 2022] POST /identity/v3/users => generated 286 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:45.998590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:46.002092 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:46.118021 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d14f5398f04c9b89a909d5e356b70e', 'request_id': 'req-77de87a6-c10b-40fe-b03d-59b24fa54726'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:46.141421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d14f5398f04c9b89a909d5e356b70e', 'request_id': 'req-77de87a6-c10b-40fe-b03d-59b24fa54726'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:46.168520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af1e2-c48d-48d9-b167-ad4b8b8c205b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:46.248280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:46.275140 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:46.278787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:46.279507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:46.280126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:46.408162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:46.411870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:46.429423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1194/2395] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:46.430473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:46.442542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d332-0145-4f25-ace6-a0f5305acab7 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:46.454027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:46.455081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:46.455492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:46.455918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:46.457125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:46.461052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:46.474500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1195/2396] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:46.480163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:46.501530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521de36-b5f9-47ad-97fa-e8aedc9625cc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:46.510516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:46.511693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:46.512203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:46.512771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:46.514235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:46.517487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:46.539519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1196/2397] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:28:46.545934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:46.565767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c802c5-fbce-4021-bf54-0cfc1540024f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:46.591246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:46.593742 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:46.594284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:46.594789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] PATH_INFO: `/v3/projects/25d416b8ef5249fdae51f9b42a73be90/users/91f40fa394354b9a961d73e65e0bd6ec/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:46.643720 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d416b8ef5249fdae51f9b42a73be90, user_id=91f40fa394354b9a961d73e65e0bd6ec, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:46.650474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c00afc-4202-4a31-8d5e-a05d862bd590 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:46.916980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d14f5398f04c9b89a909d5e356b70e', 'request_id': 'req-77de87a6-c10b-40fe-b03d-59b24fa54726'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:46.932577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] The token cache is being invalidate because project 48d14f5398f04c9b89a909d5e356b70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:46.933140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:46.933937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-77de87a6-c10b-40fe-b03d-59b24fa54726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:46.936078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1202/2398] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:45 2022] DELETE /identity/v3/projects/48d14f5398f04c9b89a909d5e356b70e => generated 0 bytes in 1184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:46.987034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c08f77-8642-4774-a030-28017ac24ddd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:46.987034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c08f77-8642-4774-a030-28017ac24ddd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:46.987034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c08f77-8642-4774-a030-28017ac24ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.000006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1197/2399] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:46 2022] PUT /identity/v3/projects/25d416b8ef5249fdae51f9b42a73be90/users/91f40fa394354b9a961d73e65e0bd6ec/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:47.095659 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-76c08f77-8642-4774-a030-28017ac24ddd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:47.097233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-76c08f77-8642-4774-a030-28017ac24ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:47.116818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.117803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.118607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.119152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:47.204948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] MFA Rules not processed for user `91f40fa394354b9a961d73e65e0bd6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:47.219199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:47.277332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1203/2400] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:47.291252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.291810 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.292362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.326651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:47.327922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:47.339232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1204/2401] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:47.345548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:47.359539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c01064-0f1f-4bac-9a60-bd2fa33c0623 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:47.366305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:47.367094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.367467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.367841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.368704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:47.369077 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:47.370820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:47.394441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1205/2402] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:28:47.397276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:47.410758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a22b3-db6a-42fd-a854-bba57f5f8a1e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:47.417144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:47.419763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.419763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.419763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.419763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:47.422307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:47.431770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1198/2403] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:47.438275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:47.442743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:47.562980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:47.577489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1206/2404] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:47 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:47.579912 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:47.643948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bcbfe-6fa5-4283-8013-cb13827d9f13 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:47.771726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180eeb8a-9c02-462f-b130-5ab91daecd9d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:47.782587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:47.785064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.785064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.785064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.785064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:47.790134 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:47.823579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:47.828808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.828808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.828808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.920653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1207/2405] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:47 2022] POST /identity/v3/users => generated 287 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:28:47.922904 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:47.927356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:47.947220 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a652783-1d23-4aaa-a45c-5f3487060980 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:47.950009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:47.952449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:47.953635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:47.954892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:47.954892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] PATH_INFO: `/v3/projects/f36dd5b66ec6412c90f3d5384285feeb` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:47.955462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:47.968690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36dd5b66ec6412c90f3d5384285feeb)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:47.974645 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:47.993837 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1199/2406] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:47.994496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:48.016201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f9bdf-3363-4781-bbdd-314b7698eb13 service glance] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:48.029012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:48.038363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:48.038363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:48.038363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:48.038363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:48.038363 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:48.061671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1200/2407] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:48.075814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:48.107420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259380b9-4014-458b-9288-4207e7fba0f9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:48.116941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:48.117744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:48.119122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:48.119676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:48.123937 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:48.127964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:48.158172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1201/2408] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:48.163500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:48.178450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdd5a0-9061-475c-b05a-5438ade1b3fd admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:48.222603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36dd5b66ec6412c90f3d5384285feeb', 'request_id': 'req-0f041428-c60e-41c5-8554-1fa85254840f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:48.238016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36dd5b66ec6412c90f3d5384285feeb', 'request_id': 'req-0f041428-c60e-41c5-8554-1fa85254840f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:48.301656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:48.304952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:48.306578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:48.307195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] PATH_INFO: `/v3/projects/a5fb38686f44418daa614689ecdc00d8/users/26a6795b0f9b4d4983441bfedf4341df/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:48.319654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36dd5b66ec6412c90f3d5384285feeb', 'request_id': 'req-0f041428-c60e-41c5-8554-1fa85254840f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:48.321503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:48.349545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] The token cache is being invalidate because project f36dd5b66ec6412c90f3d5384285feeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:48.350332 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:48.354396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0f041428-c60e-41c5-8554-1fa85254840f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:48.355508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fb38686f44418daa614689ecdc00d8, user_id=26a6795b0f9b4d4983441bfedf4341df, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:48.368372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1208/2409] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:47 2022] DELETE /identity/v3/projects/f36dd5b66ec6412c90f3d5384285feeb => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:48.370145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:48.477514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1202/2410] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:48 2022] PUT /identity/v3/projects/a5fb38686f44418daa614689ecdc00d8/users/26a6795b0f9b4d4983441bfedf4341df/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:28:48.497955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:48.498258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:48.498535 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:48.636224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] MFA Rules not processed for user `26a6795b0f9b4d4983441bfedf4341df`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:48.647879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:48.802745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1209/2411] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:49.591036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:49.593735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:49.616154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:49.737809 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f2a8f-1a49-485a-a3aa-021c4a225608 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:49.752935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:49.753636 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:49.753928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:49.755509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:49.768000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:49.778118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc3d9a-f359-4011-8798-920ace5a0d8d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:49.782312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1203/2412] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:28:55.850326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:55.852266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:55.946685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c987f66d-7157-46ff-b6fe-ea87bec3387a None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:55.958250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:55.960632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:55.960901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:55.961221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] PATH_INFO: `/v3/users/26a6795b0f9b4d4983441bfedf4341df` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:55.962637 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26a6795b0f9b4d4983441bfedf4341df)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:55.965130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:56.172957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26a6795b0f9b4d4983441bfedf4341df'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:56.197184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26a6795b0f9b4d4983441bfedf4341df'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:56.216746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26a6795b0f9b4d4983441bfedf4341df'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:56.218997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26a6795b0f9b4d4983441bfedf4341df'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:56.220255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26a6795b0f9b4d4983441bfedf4341df'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:56.297169 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:56.304089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1210/2413] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:28:55 2022] DELETE /identity/v3/users/26a6795b0f9b4d4983441bfedf4341df => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:56.309302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:56.309934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:56.311385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:56.336496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:56.337751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:56.416936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1204/2414] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:28:56.428621 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:56.430268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:56.534176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ec663-7931-4f62-a1d6-d349db6dc7c4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:56.541131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:56.542337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:56.542849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:56.543420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:56.557984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:56.560772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e44e8-25f9-43ef-9036-1402202a5941 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:56.573102 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1211/2415] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:28:57.201726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:57.206644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:57.216874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c91c0b-457a-4d37-bbb6-e9c1b39a8512 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:57.225796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:57.228119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.228119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.244982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] PATH_INFO: `/v3/projects/a5fb38686f44418daa614689ecdc00d8` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.244982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fb38686f44418daa614689ecdc00d8)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:57.244982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:57.380806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fb38686f44418daa614689ecdc00d8', 'request_id': 'req-71752076-e116-4a73-b98e-64449d148a33'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:57.388116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fb38686f44418daa614689ecdc00d8', 'request_id': 'req-71752076-e116-4a73-b98e-64449d148a33'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:57.490647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fb38686f44418daa614689ecdc00d8', 'request_id': 'req-71752076-e116-4a73-b98e-64449d148a33'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:57.503068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] The token cache is being invalidate because project a5fb38686f44418daa614689ecdc00d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:28:57.503369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:57.504057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-71752076-e116-4a73-b98e-64449d148a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:28:57.506251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1205/2416] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:28:57 2022] DELETE /identity/v3/projects/a5fb38686f44418daa614689ecdc00d8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:57.511269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.512028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.512028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.531570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:57.532840 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:57.597342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1212/2417] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:57.605625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.606012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.606302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.632595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:57.634802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:57.644717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1206/2418] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:57.651199 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:57.666078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ded94-3728-42db-99f6-48a4ca95f912 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:57.671266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:57.672060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.672464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.672847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.673602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:28:57.673914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:57.675625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:57.689767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1213/2419] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:28:57.699811 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:57.716756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c4baa-84bb-4e1f-be8b-b307bd071864 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:57.731797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:57.733037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.733543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.734099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.735463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:57.739472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:57.839465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1207/2420] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:57 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:57.844017 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:57.855662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed9f0a-a0ac-4b5a-ab1e-a26c85642985 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:57.943305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:57.944075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:57.944400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:57.944684 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:57.945421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:57.947378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.105901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1214/2421] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:57 2022] POST /identity/v3/users => generated 292 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:58.111020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.117969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:58.121942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd043e0c-a287-464e-a858-5d2f15d108c8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.126673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.127343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.127629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.127908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.128718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.130897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.142858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1208/2422] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:58.148089 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.158927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9f607-2cae-4a68-a0de-805158769a9f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.165493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.165493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.165493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.165942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.166555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.168159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27cf92-0898-4c87-8652-9491a1ff861e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.179560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1215/2423] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:58.183900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.195682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045fd74-3096-4c86-be9b-0746952c4b35 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.200567 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.201530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.201820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.202103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] PATH_INFO: `/v3/projects/88a9399b074a49089d44d6f349a22cf6/users/9e7102db12164fa594d23cd50994b119/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.219806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=88a9399b074a49089d44d6f349a22cf6, user_id=9e7102db12164fa594d23cd50994b119, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.227918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.305921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1209/2424] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:58 2022] PUT /identity/v3/projects/88a9399b074a49089d44d6f349a22cf6/users/9e7102db12164fa594d23cd50994b119/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:28:58.381462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.381825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.382229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.396751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:58.414610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] MFA Rules not processed for user `9e7102db12164fa594d23cd50994b119`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:58.415859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:58.506884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1216/2425] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:58.512169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.590333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f18b4-6a98-4c7d-b6b9-80b1b11df7ac admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.595978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.597059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.597480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.597856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.598744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.600854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.698510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1210/2426] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:58 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:58.705593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.716544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78ef24-9b8e-4899-9b03-2c8cb3ede59e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.777071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.777755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.778038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.778339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.779044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.781041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.870642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1217/2427] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:58 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:58.872465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.883369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62457d-2812-43c7-a314-13a8bdc13a80 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.891519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.891519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.891519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.891519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.891957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.892133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.907285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1211/2428] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:58.912231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.929007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485e9be-b89b-4d58-91b1-e0d7519d093e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.937603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.938910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.939481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.940000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:58.941585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:58.945277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181d4c7f-7375-4087-9871-ee8af416a38b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:58.961880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1218/2429] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:58.966849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:58.978079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef2ae0-f2cc-4aef-8122-b51213beecb4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:58.983118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:58.984870 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:58.985165 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:58.985451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] PATH_INFO: `/v3/projects/ff4abb1f6d944184a7ee70537cbdb891/users/9041e4af0a7b4e4e967bb8b5a26264a7/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.003934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4abb1f6d944184a7ee70537cbdb891, user_id=9041e4af0a7b4e4e967bb8b5a26264a7, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.006972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.076630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1212/2430] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:58 2022] PUT /identity/v3/projects/ff4abb1f6d944184a7ee70537cbdb891/users/9041e4af0a7b4e4e967bb8b5a26264a7/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:59.089781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.090075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.090357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.121995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] MFA Rules not processed for user `9041e4af0a7b4e4e967bb8b5a26264a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:28:59.123240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:28:59.205781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1219/2431] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:28:59.212976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.293920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87789a-61cb-4acf-9569-8e926763bcff admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.301860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.302576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.302867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.303154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.303871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.306212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.396741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1213/2432] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:28:59 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:28:59.402639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.409752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:59.415059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17638d2-20ea-48b9-afcb-367d1d03f549 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.489159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.489853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.490137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.490425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.491159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.493248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.573130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1220/2433] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:28:59 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:28:59.577975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.586714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:28:59.592750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb021c-ecef-4939-921c-371376c779b4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.600733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.601444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.601748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.602042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.602797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.604514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.616734 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1214/2434] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:59.621271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.633245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5dc15-971e-41fb-9078-9b431c78b4a8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.638064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.638971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.639259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.639537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] PATH_INFO: `/v3/projects/6a8313e363604f34a552ecee57d01b72/users/783e53dd94c4408fbb63f7992245a82a/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.659636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8313e363604f34a552ecee57d01b72, user_id=783e53dd94c4408fbb63f7992245a82a, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.662675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.743983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1221/2435] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:59 2022] PUT /identity/v3/projects/6a8313e363604f34a552ecee57d01b72/users/783e53dd94c4408fbb63f7992245a82a/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:59.749210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.764693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017577a4-c411-47b0-bea5-bcbf6f215499 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.861930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.863066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.863572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.864751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.866219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.869202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba01af-09ed-437f-97c6-1855062ad3aa admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.895832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1215/2436] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:28:59.897767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:28:59.914302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8d69-449e-47b1-9661-3a8da7c904f8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:28:59.922025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:28:59.923090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.923497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.923993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] PATH_INFO: `/v3/projects/6a8313e363604f34a552ecee57d01b72/users/783e53dd94c4408fbb63f7992245a82a/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:28:59.927719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8313e363604f34a552ecee57d01b72, user_id=783e53dd94c4408fbb63f7992245a82a, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:28:59.931661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:28:59.946817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1222/2437] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:28:59 2022] PUT /identity/v3/projects/6a8313e363604f34a552ecee57d01b72/users/783e53dd94c4408fbb63f7992245a82a/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:28:59.959135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:28:59.959657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:28:59.960164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:00.011303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] MFA Rules not processed for user `783e53dd94c4408fbb63f7992245a82a`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:00.013576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:00.142968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1216/2438] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:28:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:00.158208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:00.160094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:00.225546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ef2ca-aed3-45d4-a012-cee498ec7cec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:00.230208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:00.230889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:00.231173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:00.231452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:00.305382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:00.306893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:00.317537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1223/2439] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:02.389886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:02.400875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c25bf6-3392-4bbe-8fa7-2801b6ddaa37 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:02.405292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:02.412224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:02.412224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:02.412224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:02.478751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:02.482103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:02.498319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1217/2440] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:04.336597 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:04.338255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:04.351564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac7e0c-f24c-46f4-9d8c-44873891739a service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:04.355594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:04.356255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:04.356577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:04.356856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:04.374040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:04.376378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08894245-8cba-4818-a623-bd4bc89289ac service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:04.384696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:04.395884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1224/2441] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:05.630168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:05.631725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:05.652061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:05.727373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb64093-1c69-4b69-99c6-d680f817600f service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:05.731620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:05.732333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:05.732629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:05.732968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:05.735622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:05.887382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:05.889139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:05.902660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1218/2442] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:07.936981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:07.937500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:07.937699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:07.960749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:07.962132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:08.042574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1225/2443] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:29:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:08.059190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:08.060950 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:08.155969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342b71b-0cc1-46cb-b9fd-4b8caba5979b service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:08.163014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:08.163014 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:08.163518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:08.163645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:08.174016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:08.175957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfebbb0b-f333-438a-a80b-7311f17ccaf6 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:08.185568 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1219/2444] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:29:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:29:09.408828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:09.621291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be32d0a-4d60-49bd-83f3-b3807ef7bb81 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:09.632382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:09.634087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:09.634595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:09.635060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] PATH_INFO: `/v3/users/9e7102db12164fa594d23cd50994b119` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:09.640656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7102db12164fa594d23cd50994b119)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:09.643442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:09.955327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7102db12164fa594d23cd50994b119'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:09.981728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7102db12164fa594d23cd50994b119'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:10.031639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7102db12164fa594d23cd50994b119'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:10.032857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7102db12164fa594d23cd50994b119'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:10.034198 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7102db12164fa594d23cd50994b119'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:10.162151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1226/2445] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:29:09 2022] DELETE /identity/v3/users/9e7102db12164fa594d23cd50994b119 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:10.166534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:10.167194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:10.167194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:10.210242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:10.211538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:10.331179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:10.355592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1220/2446] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:29:10.370614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:10.372595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:10.409326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:10.461221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d120808-45c8-4b89-938b-9a9ceb8001ce admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:10.465436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:10.466115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:10.466439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:10.466726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:10.478531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:10.480753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:10.493952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1227/2447] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:29:11.022702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:11.025008 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:11.055706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd573b-3ed3-4ff2-9fa5-69840395e5e7 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:11.066484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:11.069213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:11.070410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:11.071902 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] PATH_INFO: `/v3/projects/88a9399b074a49089d44d6f349a22cf6` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:11.074398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=88a9399b074a49089d44d6f349a22cf6)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:11.078893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:11.257526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88a9399b074a49089d44d6f349a22cf6', 'request_id': 'req-e4e77227-7041-47b7-8b5f-267aa2b39bee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.264205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88a9399b074a49089d44d6f349a22cf6', 'request_id': 'req-e4e77227-7041-47b7-8b5f-267aa2b39bee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.330524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88a9399b074a49089d44d6f349a22cf6', 'request_id': 'req-e4e77227-7041-47b7-8b5f-267aa2b39bee'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.343044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] The token cache is being invalidate because project 88a9399b074a49089d44d6f349a22cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:29:11.343337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.344111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.345721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1221/2448] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:29:11 2022] DELETE /identity/v3/projects/88a9399b074a49089d44d6f349a22cf6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:11.350891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:11.361297 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979bdc8-630e-45f4-bcd3-a9791e068c14 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:11.426255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:11.428266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:11.428482 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:11.428786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] PATH_INFO: `/v3/users/9041e4af0a7b4e4e967bb8b5a26264a7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:11.430205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9041e4af0a7b4e4e967bb8b5a26264a7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:11.432166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:11.686337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9041e4af0a7b4e4e967bb8b5a26264a7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.693954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9041e4af0a7b4e4e967bb8b5a26264a7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.708586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9041e4af0a7b4e4e967bb8b5a26264a7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.709355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9041e4af0a7b4e4e967bb8b5a26264a7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.709973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9041e4af0a7b4e4e967bb8b5a26264a7'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:11.816382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1228/2449] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:29:11 2022] DELETE /identity/v3/users/9041e4af0a7b4e4e967bb8b5a26264a7 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:29:12.299747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:12.302994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:12.340662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e77227-7041-47b7-8b5f-267aa2b39bee admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:12.462961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:12.468155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:12.468155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:12.468155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] PATH_INFO: `/v3/projects/ff4abb1f6d944184a7ee70537cbdb891` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:12.468155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4abb1f6d944184a7ee70537cbdb891)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:12.469643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:12.593604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4abb1f6d944184a7ee70537cbdb891', 'request_id': 'req-102b0eb4-380e-47e4-96b7-a2d375292cc3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:12.604333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4abb1f6d944184a7ee70537cbdb891', 'request_id': 'req-102b0eb4-380e-47e4-96b7-a2d375292cc3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:12.719226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4abb1f6d944184a7ee70537cbdb891', 'request_id': 'req-102b0eb4-380e-47e4-96b7-a2d375292cc3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:12.740793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] The token cache is being invalidate because project ff4abb1f6d944184a7ee70537cbdb891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:29:12.741310 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:12.742414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:12.745860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1222/2450] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:29:12 2022] DELETE /identity/v3/projects/ff4abb1f6d944184a7ee70537cbdb891 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:29:12.749859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:12.752382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:12.769446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c48ba3-cb93-4ad7-9b29-f31669a26de0 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:12.825064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:12.845033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:12.846000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:12.846432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:12.846954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] PATH_INFO: `/v3/users/783e53dd94c4408fbb63f7992245a82a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:12.848329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=783e53dd94c4408fbb63f7992245a82a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:12.850628 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:13.058427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783e53dd94c4408fbb63f7992245a82a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:13.069409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783e53dd94c4408fbb63f7992245a82a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:13.088027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783e53dd94c4408fbb63f7992245a82a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:13.088832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783e53dd94c4408fbb63f7992245a82a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:13.089466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a158e129-ec99-492a-9391-975a43064579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783e53dd94c4408fbb63f7992245a82a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:13.218653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1229/2451] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:29:12 2022] DELETE /identity/v3/users/783e53dd94c4408fbb63f7992245a82a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:29:13.771882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:13.773444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:13.788464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b0eb4-380e-47e4-96b7-a2d375292cc3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:13.792692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:13.857390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:13.858277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:13.858573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:13.858865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] PATH_INFO: `/v3/projects/6a8313e363604f34a552ecee57d01b72` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:13.860499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8313e363604f34a552ecee57d01b72)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:13.864991 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.010822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8313e363604f34a552ecee57d01b72', 'request_id': 'req-d858c0d4-eaf4-4801-b10b-fa037689ae04'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:14.017799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8313e363604f34a552ecee57d01b72', 'request_id': 'req-d858c0d4-eaf4-4801-b10b-fa037689ae04'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:14.078434 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8313e363604f34a552ecee57d01b72', 'request_id': 'req-d858c0d4-eaf4-4801-b10b-fa037689ae04'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:14.096667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] The token cache is being invalidate because project 6a8313e363604f34a552ecee57d01b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:29:14.096963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:14.097663 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-d858c0d4-eaf4-4801-b10b-fa037689ae04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:14.099781 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1223/2452] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:29:13 2022] DELETE /identity/v3/projects/6a8313e363604f34a552ecee57d01b72 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:14.104807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.105321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.105430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.126094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:14.127347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:14.200363 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1230/2453] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:14.209722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.210026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.213283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.229775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:14.231070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:14.240433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1224/2454] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:14.244586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.254475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7c6e0-d6ca-4a86-a4b4-6efbe0d5ee6b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.259844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.260547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.260848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.261136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.261889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:29:14.262209 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.263838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.276051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1231/2455] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:29:14.280561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.290940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd505fb-e0f1-4717-8845-9b0dbc274536 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.295973 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.296664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.296957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.297239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.297943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.299879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.381395 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1225/2456] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:29:14 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:29:14.385736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.397117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc04b02-25c0-43a3-b7e3-b4601df721a4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.459276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:14.469051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.469738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.470028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.470312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.471033 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.473106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.589164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1232/2457] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:29:14 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:29:14.595803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.607863 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0ef2-b676-4c7a-be5b-00f7cc3912f6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.613078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.613788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.615235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.615533 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.616292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.617960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.633157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1226/2458] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:29:14.637242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.649579 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a296e-e9b2-49a8-b68a-e69fcede9f54 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.654400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.655079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.655378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.655658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.656440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.658073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d427fbc-4798-45fa-b161-a6fe8315eeca admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.669754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1233/2459] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:29:14.674314 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.693648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086fad-886f-49b8-8da6-2b1d83fc7bb4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:14.702436 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:14.704733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.704909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.705501 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] PATH_INFO: `/v3/projects/0a19002b73c84cd09072276f1b67a772/users/06bf03defaed4b1988a5a5c38984c80a/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.735198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a19002b73c84cd09072276f1b67a772, user_id=06bf03defaed4b1988a5a5c38984c80a, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:14.742142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:14.819900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1227/2460] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:29:14 2022] PUT /identity/v3/projects/0a19002b73c84cd09072276f1b67a772/users/06bf03defaed4b1988a5a5c38984c80a/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:14.831418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:14.831716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:14.832188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:14.863369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] MFA Rules not processed for user `06bf03defaed4b1988a5a5c38984c80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:14.865173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:14.933034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1234/2461] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:14.944574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:14.946359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:14.985326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:15.009033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b27e88-ff29-4a90-904e-937cc1c25e8f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:15.012909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:15.013571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:15.013860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:15.014142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:15.023876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:15.025394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f97fa-4be4-44e2-9f8b-2f9cc780292b service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:15.034975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1228/2462] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:22.061865 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:22.063530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:22.150445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c728a2-3f3d-45e3-bad8-5f45302704ae None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:22.155531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:22.157107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:22.157829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:22.158335 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] PATH_INFO: `/v3/users/91f40fa394354b9a961d73e65e0bd6ec` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:22.159882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f40fa394354b9a961d73e65e0bd6ec)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:22.162689 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:22.421121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f40fa394354b9a961d73e65e0bd6ec'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:22.429383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f40fa394354b9a961d73e65e0bd6ec'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:22.434382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:22.446393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f40fa394354b9a961d73e65e0bd6ec'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:22.447257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f40fa394354b9a961d73e65e0bd6ec'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:22.448067 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f40fa394354b9a961d73e65e0bd6ec'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:22.550856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1235/2463] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:29:22 2022] DELETE /identity/v3/users/91f40fa394354b9a961d73e65e0bd6ec => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:29:22.556071 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:22.556642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:22.556833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:22.575909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:22.578238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:22.643472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1229/2464] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:22.653523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:22.654968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:22.724736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2fcff-6b0f-4b38-8570-f1b262f0effe admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:22.728449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:22.729327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:22.729470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:22.729666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:22.744581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:22.744581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c55503-fb2e-46c8-ae87-d4a23b0fc365 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:22.756057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1236/2465] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:23.259993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:23.261473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:23.274093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ae30f-748e-448c-ae84-f5d3b26393fa None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:23.279114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:23.280388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:23.280758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:23.281037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] PATH_INFO: `/v3/projects/25d416b8ef5249fdae51f9b42a73be90` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:23.283647 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d416b8ef5249fdae51f9b42a73be90)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:23.285820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:23.434336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d416b8ef5249fdae51f9b42a73be90', 'request_id': 'req-85052049-4213-42a8-b847-9dd2381209a8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:23.441367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d416b8ef5249fdae51f9b42a73be90', 'request_id': 'req-85052049-4213-42a8-b847-9dd2381209a8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:23.721255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d416b8ef5249fdae51f9b42a73be90', 'request_id': 'req-85052049-4213-42a8-b847-9dd2381209a8'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:23.736002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] The token cache is being invalidate because project 25d416b8ef5249fdae51f9b42a73be90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:29:23.736483 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:23.737169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:23.738752 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1230/2466] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:29:23 2022] DELETE /identity/v3/projects/25d416b8ef5249fdae51f9b42a73be90 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:26.868498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c808d-d1a3-4f08-b032-46757296d95a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:26.869030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c808d-d1a3-4f08-b032-46757296d95a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:26.869157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c808d-d1a3-4f08-b032-46757296d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:26.887257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c44c808d-d1a3-4f08-b032-46757296d95a None None] MFA Rules not processed for user `b58b0d520aa1418bb6ad3b5645a826a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:26.888522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c44c808d-d1a3-4f08-b032-46757296d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:26.941600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:26.942791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:26.948614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1237/2467] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:27.008577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85052049-4213-42a8-b847-9dd2381209a8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:27.014928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:27.014928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:27.014928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:27.014928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] PATH_INFO: `/v3/users/06bf03defaed4b1988a5a5c38984c80a` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:27.015826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bf03defaed4b1988a5a5c38984c80a)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:27.017939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:27.019552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:27.231908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bf03defaed4b1988a5a5c38984c80a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:27.232549 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bf03defaed4b1988a5a5c38984c80a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:27.246348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bf03defaed4b1988a5a5c38984c80a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:27.252897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bf03defaed4b1988a5a5c38984c80a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:27.318700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bf03defaed4b1988a5a5c38984c80a'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:27.326298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1231/2468] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:29:26 2022] DELETE /identity/v3/users/06bf03defaed4b1988a5a5c38984c80a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:29:27.331307 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:27.333459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:27.333830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:27.353716 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:29:27.354950 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:27.422277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1238/2469] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:29:27.433113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:27.434826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:27.506665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a3acb-3435-4733-bf68-cebb88e88140 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:27.510575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:27.511659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:27.511659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:27.511881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:27.521770 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:27.523274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79456e-74d7-407f-967b-68927f1eec40 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:27.533876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1232/2470] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:29:28.132331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:29:28.133796 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:29:28.146951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d96c5b-a80e-4a61-bd77-30711c8877ba None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:29:28.152002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:29:28.152907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:29:28.153206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:29:28.153487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] PATH_INFO: `/v3/projects/0a19002b73c84cd09072276f1b67a772` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:29:28.154700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a19002b73c84cd09072276f1b67a772)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:29:28.156897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:29:28.180561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:28.257091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a19002b73c84cd09072276f1b67a772', 'request_id': 'req-c6c915d4-0293-4c59-8955-261874286e73'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:28.264426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a19002b73c84cd09072276f1b67a772', 'request_id': 'req-c6c915d4-0293-4c59-8955-261874286e73'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:28.352858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a19002b73c84cd09072276f1b67a772', 'request_id': 'req-c6c915d4-0293-4c59-8955-261874286e73'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:28.364892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] The token cache is being invalidate because project 0a19002b73c84cd09072276f1b67a772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:29:28.365188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:28.365867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:29:28.367451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1239/2471] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:29:28 2022] DELETE /identity/v3/projects/0a19002b73c84cd09072276f1b67a772 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:29:37.529270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:29:38.361197 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:01.029323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c62e7-4f3b-4839-84cc-41c55ae961f5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:01.029864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c62e7-4f3b-4839-84cc-41c55ae961f5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:01.029970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c62e7-4f3b-4839-84cc-41c55ae961f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:01.048820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1f0c62e7-4f3b-4839-84cc-41c55ae961f5 None None] MFA Rules not processed for user `d5148114d71b401ca20f988485fb6c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:01.050063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f0c62e7-4f3b-4839-84cc-41c55ae961f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:01.116919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1233/2472] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:09.071746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:09.073416 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:09.134976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c915d4-0293-4c59-8955-261874286e73 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:09.138661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:09.139318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:09.139611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:09.139898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:09.141831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:09.220819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:09.222507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:09.235472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1240/2473] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:10.319121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:10.319691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:10.319691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:10.340338 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] MFA Rules not processed for user `29bc0d7e30794a6abb292dc65329ed9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:10.344209 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:10.407215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1234/2474] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:10.453170 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:10.455127 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:10.535586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dae923-3d13-469a-a1b2-cf5bd69e0e29 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:10.540496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:10.541172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:10.541465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:10.541745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:10.543638 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:10.566304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:10.569382 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:10.586201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1241/2475] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:16.094057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:16.096968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:16.117987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986d859-f488-4559-b74d-b64ddcf68cd5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:16.125920 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:16.127308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:16.127788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:16.128379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:16.132417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:16.224850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:16.226575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:16.237247 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1235/2476] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:20.578233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=80 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:25.473891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:25.485361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b42a90-5e4a-4478-b315-b84e35781d2c service glance] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:25.492927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:25.493593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:25.493881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:25.494168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:25.496173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:25.507908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:25.509570 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:25.519070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1242/2477] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:26.231910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=78 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:34.892172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:34.893860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:34.906951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515dc4a-d2c0-457e-a032-bb065003d7f5 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:34.911290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:34.911961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:34.912249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:34.912563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:34.914462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:34.924996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:34.932685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:34.936212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1236/2478] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:35.514927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:37.938015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:37.944682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b9462-0644-4615-b07d-aafed9bc12db service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:37.949563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:37.950228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:37.950511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:37.950783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:37.953323 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:38.029073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:38.030531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:38.042106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1243/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:42.608749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:42.610709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:42.629004 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2b4e-7d03-4452-bac9-63e96eb7fbd0 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:42.634306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:42.635658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:42.635953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:42.636235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] PATH_INFO: `/v3/users/b58b0d520aa1418bb6ad3b5645a826a7` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:42.637747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58b0d520aa1418bb6ad3b5645a826a7)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:42.639929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:42.840386 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58b0d520aa1418bb6ad3b5645a826a7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:42.841289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58b0d520aa1418bb6ad3b5645a826a7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:42.859702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58b0d520aa1418bb6ad3b5645a826a7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:42.870802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58b0d520aa1418bb6ad3b5645a826a7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:42.947947 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6294e945-7545-46ef-90a4-2263902adcda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58b0d520aa1418bb6ad3b5645a826a7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:42.956612 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1237/2480] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:30:42 2022] DELETE /identity/v3/users/b58b0d520aa1418bb6ad3b5645a826a7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:43.289994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:43.304604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed48667-ee87-4226-baca-31526c9b0b8f service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:43.381547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:43.382465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:43.382757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:43.383045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] PATH_INFO: `/v3/projects/54167aa5315d4dd2b514ad4b027634f9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:43.385058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] RBAC: Authorizing `identity:delete_project(project_id=54167aa5315d4dd2b514ad4b027634f9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:43.387255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:43.510496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54167aa5315d4dd2b514ad4b027634f9', 'request_id': 'req-7ef3ac12-013e-4089-95f7-10bc3955b33f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:43.518349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54167aa5315d4dd2b514ad4b027634f9', 'request_id': 'req-7ef3ac12-013e-4089-95f7-10bc3955b33f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:43.595413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54167aa5315d4dd2b514ad4b027634f9', 'request_id': 'req-7ef3ac12-013e-4089-95f7-10bc3955b33f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:43.612407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] The token cache is being invalidate because project 54167aa5315d4dd2b514ad4b027634f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:30:43.613050 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:43.614106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7ef3ac12-013e-4089-95f7-10bc3955b33f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:43.616217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1244/2481] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:30:43 2022] DELETE /identity/v3/projects/54167aa5315d4dd2b514ad4b027634f9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:43.621221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:43.621702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:43.621787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:43.645871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:43.647558 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:43.736497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1238/2482] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:30:43.739523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:43.739965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:43.740349 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:43.760800 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:43.762526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:43.773065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1245/2483] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:30:43.777093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:43.789953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b053b41-585c-4a34-88c6-0f53357c5636 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:43.796326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:43.797127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:43.797586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:43.798019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:43.798961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:30:43.799270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:43.800965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:43.814903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1239/2484] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:30:43.825263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:43.839589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263ac518-2b7d-4f30-a726-f4dc30172258 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:43.844940 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:43.845735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:43.846080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:43.846419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:43.847532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:43.850056 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:43.867373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:44.028766 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1246/2485] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:30:43 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:30:44.036250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:44.047629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f23db-71f6-4664-8ebf-a859bf9ab0a1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:44.145184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:44.147759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:44.305221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746ac34-56ab-44ea-b058-64df2a27583a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:44.317466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:44.318440 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.318914 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:44.319379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:44.322445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:44.353554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:44.355352 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:44.382207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1247/2486] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:44.383500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:44.408346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:44.408346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.408346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:44.408346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:44.409137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:44.415096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:44.560788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:44.562806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1240/2487] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:30:44 2022] POST /identity/v3/users => generated 283 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:30:44.588160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2bf49-c202-48f4-9e89-85a41ab08602 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:44.597572 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:44.598137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.598486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:44.599289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:44.601955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:44.603888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:44.628446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:44.639339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1248/2488] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:30:44.660162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8388d-e560-4c6f-baf3-ff044740dd29 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:44.670957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:44.672219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.681115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:44.683312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:44.686951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:44.698164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e74afc-2e06-49dc-bc93-174994d6df5d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:44.746510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1241/2489] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:30:44.751545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:44.767936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd0ecb-b036-447f-b030-661ab6170856 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:44.777214 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:44.778787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.779151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:44.779508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] PATH_INFO: `/v3/projects/aaa48715be3f4956b1fb346605c6f300/users/80998eb571fa4ec8ae500a3ff0ed4335/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:44.799354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa48715be3f4956b1fb346605c6f300, user_id=80998eb571fa4ec8ae500a3ff0ed4335, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:44.802203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:44.984429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1249/2490] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:30:44 2022] PUT /identity/v3/projects/aaa48715be3f4956b1fb346605c6f300/users/80998eb571fa4ec8ae500a3ff0ed4335/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:44.996121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:44.996701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:45.000579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:45.038832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] MFA Rules not processed for user `80998eb571fa4ec8ae500a3ff0ed4335`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:45.040217 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:45.141539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1242/2491] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:45.155616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:45.157648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:45.248867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2e80-e11f-46a4-9042-b79914d053ba admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:45.254395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:45.255099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:45.255376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:45.256556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:45.266852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:45.268539 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a9362-ffc0-47f3-9af0-9e66172ac3b2 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:45.296805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1250/2492] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:30:52.934665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:52.936115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:53.020588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6048-08e5-4c6a-b6c2-183e2e7cc2d5 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:53.026544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:53.027214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:53.027495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:53.027776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:53.029706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:53.044667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:53.046282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:53.058205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1243/2493] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:55.212919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:55.213489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:55.213489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:55.232164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:55.233460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:55.285563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:55.297490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1251/2494] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:30:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:55.444132 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:55.445573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:55.488656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:55.513624 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8809798-2b3f-4501-bdc7-160d08fc3576 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:55.518959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:55.520118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:55.520522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:55.520869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:55.529792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:55.531396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe83a1-9f65-4423-aa9a-d8f8919f89ad service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:55.540161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1244/2495] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:30:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:30:55.639911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:55.706702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789542f-5b1f-496e-8dcc-7c8e7829db65 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:55.712419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:55.776619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:55.776619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:55.776619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] PATH_INFO: `/v3/users/80998eb571fa4ec8ae500a3ff0ed4335` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:55.776619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80998eb571fa4ec8ae500a3ff0ed4335)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:55.776619 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:56.060049 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80998eb571fa4ec8ae500a3ff0ed4335'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.068360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80998eb571fa4ec8ae500a3ff0ed4335'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.085013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80998eb571fa4ec8ae500a3ff0ed4335'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.085755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80998eb571fa4ec8ae500a3ff0ed4335'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.086406 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80998eb571fa4ec8ae500a3ff0ed4335'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.161740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1252/2496] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:30:55 2022] DELETE /identity/v3/users/80998eb571fa4ec8ae500a3ff0ed4335 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:56.166316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:56.166615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:56.166933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:56.184812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:56.186017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:56.255488 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1245/2497] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:56.265718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:56.267408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:56.333776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e58295-f1e2-41be-a684-b5bf95626995 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:56.337753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:56.338441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:56.338734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:56.339023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:56.350808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:56.352727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54a85c-f6f9-470f-8ea1-fc4a80b87794 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:56.365000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1253/2498] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:56.837733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:56.839465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:56.856730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e91333-4813-4ec2-ab80-1b1dce5752ef None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:56.861520 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:56.862484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:56.867817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:56.867817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] PATH_INFO: `/v3/projects/aaa48715be3f4956b1fb346605c6f300` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:56.867817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa48715be3f4956b1fb346605c6f300)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:56.867817 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:56.991976 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa48715be3f4956b1fb346605c6f300', 'request_id': 'req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:56.998535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa48715be3f4956b1fb346605c6f300', 'request_id': 'req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:57.148069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa48715be3f4956b1fb346605c6f300', 'request_id': 'req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:57.161000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] The token cache is being invalidate because project aaa48715be3f4956b1fb346605c6f300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:30:57.161296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:57.161987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b1a1f9d0-7a16-4a90-9705-f29402a77ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:30:57.163537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1246/2499] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:30:56 2022] DELETE /identity/v3/projects/aaa48715be3f4956b1fb346605c6f300 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:57.168623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.168997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.169292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.188702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:57.198654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:57.200225 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:57.251971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1254/2500] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:57.260512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.260808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.261089 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.280278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:57.281544 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:57.289971 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1247/2501] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:57.294345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.305303 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a614b-bf6c-462f-9ae5-331518c9f265 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.310216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.310883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.311176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.311489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.312237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:30:57.312602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.314229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.331069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1255/2502] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:30:57.332537 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.342143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60747ac6-891c-4f62-9f4b-a34d736f3452 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.347774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.347774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.348155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.348155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.348949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.350858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.494993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1248/2503] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:30:57 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:30:57.499113 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.509977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0640e-9078-4871-aa8c-d7d36a0f279c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.569576 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.570581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.571070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.571693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.572000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.573987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.770749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1256/2504] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:30:57 2022] POST /identity/v3/users => generated 281 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:30:57.771895 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.783003 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb050d2-f8bc-407e-a9ac-2982cf1af67c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.788599 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.790745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.790745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.790745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.791369 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.793039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.806478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1249/2505] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:30:57.810849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.822985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce8b4-f991-4ba2-8df1-4aee5f00c01d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.827947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.832473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.832473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.832473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.832473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.832473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d824eeb-2b5a-478b-ad42-446b90200c8d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.844176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1257/2506] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:30:57.849121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:57.859955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf92c92-a550-426c-97e0-861840c29948 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:57.865736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:57.866628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:57.866968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:57.867245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] PATH_INFO: `/v3/projects/3f00042382a349fca3a078f71b29bd0c/users/1fae508b728949f2a19ba833b1f1279c/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:57.881135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:57.886553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f00042382a349fca3a078f71b29bd0c, user_id=1fae508b728949f2a19ba833b1f1279c, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:57.889506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:57.995272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1250/2507] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:30:57 2022] PUT /identity/v3/projects/3f00042382a349fca3a078f71b29bd0c/users/1fae508b728949f2a19ba833b1f1279c/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:58.007419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.007791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.008162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.035213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] MFA Rules not processed for user `1fae508b728949f2a19ba833b1f1279c`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:58.036960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:58.104328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1258/2508] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:58.108999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.177910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7451a-e268-490d-96b8-8c5d25ddf787 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.185195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.186291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.186400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.186707 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.187376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.190173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.315901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1251/2509] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:30:58 2022] POST /identity/v3/projects => generated 365 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:30:58.316921 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.327918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ca73f-b1cb-471f-9711-b2439f491a28 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.338851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:58.389937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.390732 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.391454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.391454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.392723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.396909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.564393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1259/2510] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:30:58 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:30:58.565671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.576372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb830df8-996a-4f16-9a7c-d343eae3e706 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.581370 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.582032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.582965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.583262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.584001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.585685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.596667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1252/2511] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:30:58.600929 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.611643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdcbf6-a714-4f2c-9644-0dbba0280396 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.616721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.617611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.617898 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.618178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] PATH_INFO: `/v3/projects/ab571ec94677457c88ccee3256a11c0f/users/52ed669d213c45789d9e6c1e38d34990/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.634133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab571ec94677457c88ccee3256a11c0f, user_id=52ed669d213c45789d9e6c1e38d34990, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.636858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.738705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1260/2512] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:30:58 2022] PUT /identity/v3/projects/ab571ec94677457c88ccee3256a11c0f/users/52ed669d213c45789d9e6c1e38d34990/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:58.742732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.754993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256223b-8fea-4d18-ba69-42aec0145cea admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.844383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.845336 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.845787 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.846467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.847430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.849433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b2909-c76c-438a-8173-1926c72d5368 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.866526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1253/2513] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:30:58.871273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:58.885462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4029f9-29ee-4210-b130-74b688c8cb8c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:58.896662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:58.896662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.896662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.896662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] PATH_INFO: `/v3/projects/ab571ec94677457c88ccee3256a11c0f/users/52ed669d213c45789d9e6c1e38d34990/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.897605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab571ec94677457c88ccee3256a11c0f, user_id=52ed669d213c45789d9e6c1e38d34990, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:58.899122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:58.911934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1261/2514] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:30:58 2022] PUT /identity/v3/projects/ab571ec94677457c88ccee3256a11c0f/users/52ed669d213c45789d9e6c1e38d34990/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:30:58.925396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:58.925840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:58.926758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:58.955462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] MFA Rules not processed for user `52ed669d213c45789d9e6c1e38d34990`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:30:58.956746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:59.008390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:30:59.034986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1254/2515] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:30:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:30:59.050653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:59.053162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:59.126295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1f4a-0612-4f9f-b3c0-b00f7da21f53 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:59.132216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:59.132977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:59.133276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:59.133564 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:59.148398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:59.149919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:59.162313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1262/2516] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:30:59.353360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:30:59.355090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:59.469661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f38440-60f1-4e40-a884-23064ccdb44b None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:30:59.478055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:30:59.479094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:30:59.479797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:30:59.480257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:30:59.483086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:30:59.565195 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:30:59.568381 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb427e-674e-4c0c-a6c5-95125ccbeda3 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:30:59.586812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1255/2517] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:30:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:00.462183 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:00.647461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c4d-7a0a-4074-bfe5-60d79800601c service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:00.659368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:00.661699 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:00.662814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:00.663309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] PATH_INFO: `/v3/users/1fae508b728949f2a19ba833b1f1279c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:00.674962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fae508b728949f2a19ba833b1f1279c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:00.679672 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:00.711845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:00.995811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fae508b728949f2a19ba833b1f1279c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:01.017989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fae508b728949f2a19ba833b1f1279c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:01.050329 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fae508b728949f2a19ba833b1f1279c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:01.051927 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fae508b728949f2a19ba833b1f1279c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:01.059662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fae508b728949f2a19ba833b1f1279c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:01.198301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1263/2518] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:00 2022] DELETE /identity/v3/users/1fae508b728949f2a19ba833b1f1279c => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:01.203905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:01.204843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:01.204843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:01.223329 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:01.224613 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:01.289414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1256/2519] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:31:01.302263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:01.303943 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:01.408460 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b155925-8ee3-440e-aada-b312a1ddaee7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:01.415121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:01.415121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:01.415121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:01.416144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:01.427685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:01.429226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:01.440758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1264/2520] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:31:02.186751 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:02.189268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:02.214116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0577d5-bcf3-41f4-b702-9106471f665d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:02.226425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:02.230981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:02.230981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:02.230981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] PATH_INFO: `/v3/projects/3f00042382a349fca3a078f71b29bd0c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:02.232518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f00042382a349fca3a078f71b29bd0c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:02.236554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:02.270563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:02.403341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f00042382a349fca3a078f71b29bd0c', 'request_id': 'req-c0019fac-b3d3-445c-b191-26f970ff4e52'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.417535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f00042382a349fca3a078f71b29bd0c', 'request_id': 'req-c0019fac-b3d3-445c-b191-26f970ff4e52'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.514107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f00042382a349fca3a078f71b29bd0c', 'request_id': 'req-c0019fac-b3d3-445c-b191-26f970ff4e52'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.527557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] The token cache is being invalidate because project 3f00042382a349fca3a078f71b29bd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:02.527853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.528844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.531025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1257/2521] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:02 2022] DELETE /identity/v3/projects/3f00042382a349fca3a078f71b29bd0c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:02.536224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:02.546987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f3f97-ffa9-47eb-9df4-084ad7637857 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:02.611252 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:02.612408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:02.612700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:02.612979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] PATH_INFO: `/v3/users/52ed669d213c45789d9e6c1e38d34990` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:02.615092 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ed669d213c45789d9e6c1e38d34990)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:02.617523 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:02.775657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:02.793110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ed669d213c45789d9e6c1e38d34990'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.810133 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ed669d213c45789d9e6c1e38d34990'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.835207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ed669d213c45789d9e6c1e38d34990'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.836880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ed669d213c45789d9e6c1e38d34990'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.837964 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-6996820d-dc9a-48d6-800e-ebae005ad006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ed669d213c45789d9e6c1e38d34990'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:02.929099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1265/2522] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:02 2022] DELETE /identity/v3/users/52ed669d213c45789d9e6c1e38d34990 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:03.630630 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:03.633272 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:03.649783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0019fac-b3d3-445c-b191-26f970ff4e52 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:03.737743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:03.738928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:03.739215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:03.739495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] PATH_INFO: `/v3/projects/ab571ec94677457c88ccee3256a11c0f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:03.740963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab571ec94677457c88ccee3256a11c0f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:03.743451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:03.892992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab571ec94677457c88ccee3256a11c0f', 'request_id': 'req-28af3b42-c099-4d9b-99fb-515e01bef18e'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:03.901664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab571ec94677457c88ccee3256a11c0f', 'request_id': 'req-28af3b42-c099-4d9b-99fb-515e01bef18e'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:03.989983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab571ec94677457c88ccee3256a11c0f', 'request_id': 'req-28af3b42-c099-4d9b-99fb-515e01bef18e'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:04.002205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] The token cache is being invalidate because project ab571ec94677457c88ccee3256a11c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:04.002638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:04.003357 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-28af3b42-c099-4d9b-99fb-515e01bef18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:04.006177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1258/2523] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:03 2022] DELETE /identity/v3/projects/ab571ec94677457c88ccee3256a11c0f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:04.011233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.011765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.011989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.031868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:04.033247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:04.104087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1266/2524] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:04.112110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.112514 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.112877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.148291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:04.150094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:04.159153 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1259/2525] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:04.171618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.175346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f680d7-fd04-4b49-a30c-ccd4a7015a65 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.182231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.183295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.183455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.183847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.184751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:31:04.185481 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.187135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.204354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1267/2526] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:31:04.210930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.227124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b040c-7331-4424-be8d-6bac0f6ae1bc None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.233686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.234750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.235066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.235328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.236622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.248068 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.362249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1260/2527] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/projects => generated 355 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:31:04.364083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.383191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e92d8-3c87-44b5-8d4a-954a993ef26d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.483974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.484856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.485286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.485575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.486286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.488615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.634887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1268/2528] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/users => generated 278 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:31:04.641509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.651516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c8e69-eb6f-485c-b766-650ae198eb0d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.656479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.657245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.657468 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.657750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.658494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.660296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.662783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:04.673090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1261/2529] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:31:04.677853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.688838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85b4c2d-8fbf-4620-9fc3-3dc4fc54fdf3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.693910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.694694 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.695299 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.695521 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.696277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.697999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0c21a-e615-47d2-ac6f-62f469d03e00 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.711071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1269/2530] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:31:04.715755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:04.726737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf32a3-87f5-4bec-a98f-5dea97a5d9ca admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:04.733240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:04.733240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.733240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.733240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] PATH_INFO: `/v3/projects/e12fc9e81b474474bd60e91e502df8af/users/2af4fbeaa34d45f9aa54498110499b91/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.752415 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e12fc9e81b474474bd60e91e502df8af, user_id=2af4fbeaa34d45f9aa54498110499b91, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:04.758037 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:04.861265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:04.861852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:04.862334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:04.867980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1262/2531] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:31:04 2022] PUT /identity/v3/projects/e12fc9e81b474474bd60e91e502df8af/users/2af4fbeaa34d45f9aa54498110499b91/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:04.894688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:04.898872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] MFA Rules not processed for user `2af4fbeaa34d45f9aa54498110499b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:04.900156 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:04.974922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1270/2532] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:31:04.979266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.046411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803fb0e8-f8db-4090-a0fe-4e9ffa1f74a5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.051309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.052226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.052662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.053088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.053813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.055762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.154792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1263/2533] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:31:04 2022] POST /identity/v3/projects => generated 357 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:31:05.156876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.169123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06e00-e64e-40b2-9926-99a46a16b39f None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.239136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.239756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.240041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.240346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.241058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.242995 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.345532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1271/2534] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:31:05 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:31:05.351214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.364268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dfbb8-599c-4f4b-b09f-613daefb39ed admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.371917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.373173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.373173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.374820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.374820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.376449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.390708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1264/2535] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:31:05.396206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.411228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb2ed4-00a5-406a-bd3e-6b1e4922710d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.417301 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.418381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.418744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.419097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] PATH_INFO: `/v3/projects/c8e53253e132400aa6cb8561adbc9a87/users/ef5601d780ba4849afc488ab0ed8b02f/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.446569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e53253e132400aa6cb8561adbc9a87, user_id=ef5601d780ba4849afc488ab0ed8b02f, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.447909 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.545201 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1272/2536] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:31:05 2022] PUT /identity/v3/projects/c8e53253e132400aa6cb8561adbc9a87/users/ef5601d780ba4849afc488ab0ed8b02f/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:05.546294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.560955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f247d70-1e4e-43cd-bf4e-4edfc7a01a53 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.643259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.647081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.647836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.648341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.649274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.652181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3d478-9dc5-4004-9107-8e3d23069f37 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.669505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1265/2537] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:31:05.670393 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.683039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933c24e-037b-4cc8-a4a6-aba6fe60dc82 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.688150 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.689618 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.689908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.690419 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] PATH_INFO: `/v3/projects/c8e53253e132400aa6cb8561adbc9a87/users/ef5601d780ba4849afc488ab0ed8b02f/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.693152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e53253e132400aa6cb8561adbc9a87, user_id=ef5601d780ba4849afc488ab0ed8b02f, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.696204 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.712755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1273/2538] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:31:05 2022] PUT /identity/v3/projects/c8e53253e132400aa6cb8561adbc9a87/users/ef5601d780ba4849afc488ab0ed8b02f/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:05.722102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.722403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.723982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.757873 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] MFA Rules not processed for user `ef5601d780ba4849afc488ab0ed8b02f`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:05.759438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:05.807435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:05.851901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1266/2539] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:05 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:05.865668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:05.867553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:05.944235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bed18-c14e-40bb-9d97-c473fd70451c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:05.947949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:05.948780 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:05.949066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:05.949343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:05.960030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:05.961826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:05.966260 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:05.973848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1274/2540] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:05 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:06.227359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:06.237721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db91312-cfb5-4308-ac42-5af87b624c1d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:06.243201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:06.244139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:06.244458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:06.244746 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:06.255430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:06.257320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:06.319477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:06.321094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:06.327343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7a798-23a6-4da5-b00e-de3e1528f528 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:06.332066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:06.333557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:06.333557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:06.333557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] PATH_INFO: `/v3/users/0e36c7fd357044ccbb59bddd12b61149` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:06.334803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e36c7fd357044ccbb59bddd12b61149)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:06.336983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:06.338009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1267/2541] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:06 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:06.553657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e36c7fd357044ccbb59bddd12b61149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:06.562072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e36c7fd357044ccbb59bddd12b61149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:06.576136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e36c7fd357044ccbb59bddd12b61149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:06.577155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e36c7fd357044ccbb59bddd12b61149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:06.577666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e36c7fd357044ccbb59bddd12b61149'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:06.654031 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1275/2542] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:06 2022] DELETE /identity/v3/users/0e36c7fd357044ccbb59bddd12b61149 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:31:06.951563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:06.965860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dcf8c-86a7-4f55-aec1-5513283d6f3f service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:07.055608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:07.056530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:07.056977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:07.058057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] PATH_INFO: `/v3/projects/c9b591880e9441acac9ddf006a5f9959` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:07.058464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b591880e9441acac9ddf006a5f9959)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:07.060426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:07.201915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b591880e9441acac9ddf006a5f9959', 'request_id': 'req-bb8cacab-a910-4bc3-987c-44631d93740b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.208933 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b591880e9441acac9ddf006a5f9959', 'request_id': 'req-bb8cacab-a910-4bc3-987c-44631d93740b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.291515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b591880e9441acac9ddf006a5f9959', 'request_id': 'req-bb8cacab-a910-4bc3-987c-44631d93740b'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.297555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:07.305157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] The token cache is being invalidate because project c9b591880e9441acac9ddf006a5f9959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:07.305546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.306271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.308086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1268/2543] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:06 2022] DELETE /identity/v3/projects/c9b591880e9441acac9ddf006a5f9959 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:07.312119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:07.313611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:07.327805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5690a7-52f5-4729-a389-cdce5df293d9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:07.388599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:07.389486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:07.389770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:07.390052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] PATH_INFO: `/v3/users/4b2c7e0398cc41c18b7be1fd7fa185e0` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:07.391298 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b2c7e0398cc41c18b7be1fd7fa185e0)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:07.393288 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:07.416256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:07.581163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b2c7e0398cc41c18b7be1fd7fa185e0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.589948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b2c7e0398cc41c18b7be1fd7fa185e0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.604510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b2c7e0398cc41c18b7be1fd7fa185e0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.605266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b2c7e0398cc41c18b7be1fd7fa185e0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.609504 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7fa40309-d75d-4540-8df7-85e2928c9275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b2c7e0398cc41c18b7be1fd7fa185e0'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:07.708696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1276/2544] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:07 2022] DELETE /identity/v3/users/4b2c7e0398cc41c18b7be1fd7fa185e0 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:07.958057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:07.959611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:07.975680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8cacab-a910-4bc3-987c-44631d93740b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:08.036598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:08.037451 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:08.037740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:08.038021 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] PATH_INFO: `/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:08.039257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c64e8d93054cec8ddaff71cb6a0b39)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:08.041255 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:08.214291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c64e8d93054cec8ddaff71cb6a0b39', 'request_id': 'req-04cb2a9a-4b3f-45ac-b36b-453045b9a987'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:08.220784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c64e8d93054cec8ddaff71cb6a0b39', 'request_id': 'req-04cb2a9a-4b3f-45ac-b36b-453045b9a987'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:08.296960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c64e8d93054cec8ddaff71cb6a0b39', 'request_id': 'req-04cb2a9a-4b3f-45ac-b36b-453045b9a987'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:08.308654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] The token cache is being invalidate because project c9c64e8d93054cec8ddaff71cb6a0b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:08.308949 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:08.309618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:08.311168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1269/2545] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:07 2022] DELETE /identity/v3/projects/c9c64e8d93054cec8ddaff71cb6a0b39 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:31:15.803385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:15.803856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:15.804070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:15.825450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:15.825450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:15.890022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1277/2546] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:31:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:15.907528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:15.912791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:15.997486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb2a9a-4b3f-45ac-b36b-453045b9a987 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:16.004687 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:16.005374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:16.005677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:16.006091 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:16.014984 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:16.016959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:16.040433 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1270/2547] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:31:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:31:16.088971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:16.090764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:16.166042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7189a-e42e-4f0c-9ac8-924bf3b6a344 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:16.174483 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:16.175383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:16.175846 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:16.176723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:16.178749 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:16.194026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:16.195654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9fada-e3f0-4483-aae4-464a34ace360 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:16.210344 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1278/2548] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:26.023387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:26.202730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=8,keystone:SELECT=79,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:27.555124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:27.556658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:27.628497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb448694-f26c-4c93-b4b9-1975f7e52bb2 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:27.634763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:27.635668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:27.635959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:27.636241 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] PATH_INFO: `/v3/users/2af4fbeaa34d45f9aa54498110499b91` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:27.637553 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af4fbeaa34d45f9aa54498110499b91)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:27.639598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:27.871199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af4fbeaa34d45f9aa54498110499b91'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:27.871970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af4fbeaa34d45f9aa54498110499b91'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:27.886913 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af4fbeaa34d45f9aa54498110499b91'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:27.895029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af4fbeaa34d45f9aa54498110499b91'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:28.039699 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af4fbeaa34d45f9aa54498110499b91'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:28.046675 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1271/2549] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:27 2022] DELETE /identity/v3/users/2af4fbeaa34d45f9aa54498110499b91 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:28.053408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:28.053408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:28.053408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:28.073211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:28.074531 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:28.139234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1279/2550] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:28.149646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:28.151198 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:28.212587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344aa17-ec72-488a-9b0a-3745c6d6b55f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:28.216192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:28.216909 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:28.217193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:28.217472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:28.228194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:28.229668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:28.240901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1272/2551] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:28.762405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:28.772414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:28.782465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cbccd-2706-43dc-8e2f-75dd1cca7433 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:28.787610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:28.788553 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:28.788845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:28.789128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] PATH_INFO: `/v3/projects/e12fc9e81b474474bd60e91e502df8af` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:28.790405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e12fc9e81b474474bd60e91e502df8af)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:28.792387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:28.934425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e12fc9e81b474474bd60e91e502df8af', 'request_id': 'req-4c44d81b-d85b-4c65-8d65-84886cfc9837'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:28.940635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e12fc9e81b474474bd60e91e502df8af', 'request_id': 'req-4c44d81b-d85b-4c65-8d65-84886cfc9837'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.024035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e12fc9e81b474474bd60e91e502df8af', 'request_id': 'req-4c44d81b-d85b-4c65-8d65-84886cfc9837'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.035373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] The token cache is being invalidate because project e12fc9e81b474474bd60e91e502df8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:29.035785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.036639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.038316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1280/2552] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:28 2022] DELETE /identity/v3/projects/e12fc9e81b474474bd60e91e502df8af => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:29.042345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:29.052762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b18d-029c-4b83-a9a5-c05cd64c662d service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:29.071560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:29.109259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:29.110136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:29.110421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:29.110701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] PATH_INFO: `/v3/users/ef5601d780ba4849afc488ab0ed8b02f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:29.111903 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef5601d780ba4849afc488ab0ed8b02f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:29.114620 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:29.314449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef5601d780ba4849afc488ab0ed8b02f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.315062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef5601d780ba4849afc488ab0ed8b02f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.327133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef5601d780ba4849afc488ab0ed8b02f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.334111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef5601d780ba4849afc488ab0ed8b02f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.418374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-3b0b9710-cd58-49cf-8b88-f3bebbd2e828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef5601d780ba4849afc488ab0ed8b02f'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:29.424931 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1273/2553] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:31:29 2022] DELETE /identity/v3/users/ef5601d780ba4849afc488ab0ed8b02f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:30.064661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.067893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:30.080748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44d81b-d85b-4c65-8d65-84886cfc9837 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.145374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.148222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.148222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.148222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] PATH_INFO: `/v3/projects/c8e53253e132400aa6cb8561adbc9a87` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.148222 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e53253e132400aa6cb8561adbc9a87)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.150439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:30.270493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:30.292811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e53253e132400aa6cb8561adbc9a87', 'request_id': 'req-dbd9684b-6eea-459b-8f0e-124afb8a2a48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:30.299258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e53253e132400aa6cb8561adbc9a87', 'request_id': 'req-dbd9684b-6eea-459b-8f0e-124afb8a2a48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:30.394048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e53253e132400aa6cb8561adbc9a87', 'request_id': 'req-dbd9684b-6eea-459b-8f0e-124afb8a2a48'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:30.406104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] The token cache is being invalidate because project c8e53253e132400aa6cb8561adbc9a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:31:30.406396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:30.407081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-dbd9684b-6eea-459b-8f0e-124afb8a2a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:31:30.408712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1281/2554] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:31:30 2022] DELETE /identity/v3/projects/c8e53253e132400aa6cb8561adbc9a87 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:31:30.416494 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.417062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.417062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.435522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:30.436785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:30.500216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1274/2555] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:30.509463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.513802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.513802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.530691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:30.531877 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:30.545259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1282/2556] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:30.550171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.561340 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef0475-bfe3-43aa-8868-bf4ebc0828b0 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.566115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.566840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.567284 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.567565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.568362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:31:30.568676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.570738 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:30.583585 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1275/2557] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:31:30.588250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.597963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e101b9d-49a4-4f01-b394-a23fc3909086 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.603487 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.604152 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.604474 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.604819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.607069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.607497 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:30.704132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1283/2558] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:31:30 2022] POST /identity/v3/projects => generated 363 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:31:30.705941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.717143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae15f6-5555-42c2-aadc-80003a3b9a28 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.743712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:30.777708 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.777925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.778215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.778493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.779309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.781372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:30.909677 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1276/2559] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:31:30 2022] POST /identity/v3/users => generated 282 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:31:30.914226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.935902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c748d48-6391-4892-9a5d-109b645b3c3f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.948803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.951727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.952369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.952965 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.954727 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.959207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:30.971277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1284/2560] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:31:30.975655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:30.986637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266d21-6617-4fb0-9e7f-2a36d18ca68b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:30.992348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:30.993104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:30.993466 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:30.993876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:30.994678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:30.996344 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505b9b7-db5d-47eb-a5af-c919b3db38a7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:31.011705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1277/2561] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:31:31.013103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:31.025313 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53c61-5f8f-43d5-bf16-7645334eba27 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:31.031189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:31.032377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:31.038724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:31.039248 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] PATH_INFO: `/v3/projects/ca31703335ed407a968681a602c24117/users/6c058dd374c5449ea56911dc87d911cb/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:31.060453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca31703335ed407a968681a602c24117, user_id=6c058dd374c5449ea56911dc87d911cb, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:31.063378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:31.232788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1285/2562] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:31:31 2022] PUT /identity/v3/projects/ca31703335ed407a968681a602c24117/users/6c058dd374c5449ea56911dc87d911cb/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:31:31.251790 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:31.252161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:31.252487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:31.286129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] MFA Rules not processed for user `6c058dd374c5449ea56911dc87d911cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:31.287390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:31.369700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1278/2563] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:31:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:31.393311 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:31.401616 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:31.504005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e78e2-de60-48e4-b4ae-9ec7425fe299 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:31.509458 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:31.510128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:31.510654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:31.510935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:31.523422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:31.529505 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d0701-56f8-4a12-8c6b-894765b98283 service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:31.540782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1286/2564] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:33.400116 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:33.401673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:33.468506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99326c-bcab-4383-a717-1f44ff4cd594 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:33.472408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:33.474678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:33.474983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:33.475270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:33.477264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:33.488867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:33.491183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c82a1-b7cf-4a39-96d2-97b07f475b7f service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:33.501009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1279/2565] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:33.889484 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe405-c4c5-4389-b2ab-fd244c2263b5 None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:33.889970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe405-c4c5-4389-b2ab-fd244c2263b5 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:33.890083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fe405-c4c5-4389-b2ab-fd244c2263b5 None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:33.891557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1287/2566] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:31:33.898285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad1e8c-afdb-4715-9f52-6aad402d34e9 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:33.898582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad1e8c-afdb-4715-9f52-6aad402d34e9 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:33.898936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad1e8c-afdb-4715-9f52-6aad402d34e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:33.921998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-13ad1e8c-afdb-4715-9f52-6aad402d34e9 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:33.923401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-13ad1e8c-afdb-4715-9f52-6aad402d34e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:33.960446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:33.994245 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1280/2567] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:31:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:34.010861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fe405-c4c5-4389-b2ab-fd244c2263b5 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:34.024680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fe405-c4c5-4389-b2ab-fd244c2263b5 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:34.028563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b32ef598390145d1b88c3e3811edb79b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:34.029217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.029605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.029887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.041117 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:34.042718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daf97a-a3c0-4557-bd6b-2da7a4b21cce service glance] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:34.055410 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1288/2568] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:34.311385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.311679 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.311965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] PATH_INFO: `/` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.313385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1281/2569] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:31:34.319239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52008a-0637-48d5-9f3d-745f4481e624 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.319611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52008a-0637-48d5-9f3d-745f4481e624 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.319939 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52008a-0637-48d5-9f3d-745f4481e624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.338933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7a52008a-0637-48d5-9f3d-745f4481e624 None None] MFA Rules not processed for user `bd67da2713214172989b5c689a8885f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:34.340196 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7a52008a-0637-48d5-9f3d-745f4481e624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:34.347137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1289/2570] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:31:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:31:34.360424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:34.362312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:34.378625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942bab0-f0cd-48b7-a87e-3133a9b3c326 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:34.383639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:34.384296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.384618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.384900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.393769 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:34.395301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c7801-b949-451b-83a6-fa396ba01321 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:34.403999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1282/2571] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:34.491191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a001b-d41e-43c6-abfe-41e7045888dc None None] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.491510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a001b-d41e-43c6-abfe-41e7045888dc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.491615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a001b-d41e-43c6-abfe-41e7045888dc None None] PATH_INFO: `/` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.493472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1290/2572] 10.222.0.47 () {54 vars in 921 bytes} [Tue Jul 12 18:31:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 18:31:34.499411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.499758 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.500146 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.519748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] MFA Rules not processed for user `71160dfe2635497abc4cd1c1bffc4531`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:34.521308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:34.532098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1283/2573] 10.222.0.47 () {58 vars in 1018 bytes} [Tue Jul 12 18:31:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:34.542971 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a001b-d41e-43c6-abfe-41e7045888dc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:34.556625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a001b-d41e-43c6-abfe-41e7045888dc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:34.560472 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:34.561131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.561421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.561707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.575157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:34.577977 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf0ea2-43a1-487d-964d-c2990af345a6 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:34.590336 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1291/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:34.657686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:34.659007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:34.732373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce19bd-b7c7-41e4-9d98-89c161ed1588 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:34.741287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:34.742144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:34.742542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:34.742930 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:34.745050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:34.757904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:34.759522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:34.770390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1284/2575] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:31:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:31:37.556290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3584a-a9a1-4b92-8d89-b51674f2f02b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:37.556880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3584a-a9a1-4b92-8d89-b51674f2f02b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:37.557581 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e3584a-a9a1-4b92-8d89-b51674f2f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:37.575726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-85e3584a-a9a1-4b92-8d89-b51674f2f02b None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:37.577334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-85e3584a-a9a1-4b92-8d89-b51674f2f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:37.593612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:37.640933 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1292/2576] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:31:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:37.653274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:37.654832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:37.734692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f10e2-1794-48dd-9008-8115d1129e82 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:37.741047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:37.744944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:37.744944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:37.744944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:37.752563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:37.754795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:37.764233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1285/2577] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:31:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:31:47.713038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=16 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:47.760961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=86 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:31:54.315941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:54.316455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:54.317071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:54.338239 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:31:54.339488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:31:54.346924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1293/2578] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:31:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:31:54.362582 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:31:54.383459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d9a4d-99d1-443e-b245-1905f7a147a1 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:31:54.393120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:31:54.394860 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:31:54.396094 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:31:54.396249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:31:54.419328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:31:54.419328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:31:54.432084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1286/2579] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 18:32:04.331110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:04.371791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:04.382030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9db2ff-f669-4bf3-9361-eec85d2beb49 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:04.385823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:04.386540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:04.386848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:04.387136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:04.389013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:04.427407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:04.455418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:04.456968 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fec63-a5c0-4a5b-a68c-7a4586e282d4 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:04.467681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1294/2580] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:07.732220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:07.746448 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531253-2937-4a85-a9eb-499ec88a7fc3 service swift] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:07.753915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:07.753915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:07.753915 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:07.754759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] PATH_INFO: `/v3/users/6c058dd374c5449ea56911dc87d911cb` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:07.755897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c058dd374c5449ea56911dc87d911cb)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:07.758075 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:08.038390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c058dd374c5449ea56911dc87d911cb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.039119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c058dd374c5449ea56911dc87d911cb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.051845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c058dd374c5449ea56911dc87d911cb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.066918 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c058dd374c5449ea56911dc87d911cb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.170107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c058dd374c5449ea56911dc87d911cb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.177013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1287/2581] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:07 2022] DELETE /identity/v3/users/6c058dd374c5449ea56911dc87d911cb => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:08.181562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:08.182086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:08.182230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:08.202769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:08.204019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:08.280550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1295/2582] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:08.290813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:08.292518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:08.352236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33236e2d-74d3-4e11-b5a0-9ff54ed5b343 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:08.355845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:08.356541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:08.356818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:08.357105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:08.367206 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:08.368704 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:08.378102 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1288/2583] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:08.390655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:08.392110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:08.412461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6e2e3c-09b5-40e1-9305-5b5a0a57ebcc None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:08.417462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:08.418365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:08.418651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:08.418931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] PATH_INFO: `/v3/users/29bc0d7e30794a6abb292dc65329ed9e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:08.420174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bc0d7e30794a6abb292dc65329ed9e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:08.422171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:08.610414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bc0d7e30794a6abb292dc65329ed9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.614334 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bc0d7e30794a6abb292dc65329ed9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.629691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bc0d7e30794a6abb292dc65329ed9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.630451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bc0d7e30794a6abb292dc65329ed9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.631070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bc0d7e30794a6abb292dc65329ed9e'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:08.761069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1296/2584] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:08 2022] DELETE /identity/v3/users/29bc0d7e30794a6abb292dc65329ed9e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:08.845076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:08.846719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:08.868034 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec327ff-a9e9-419a-ad7a-c91c01aafe01 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:08.942188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:08.942693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:08.942979 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:08.943258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] PATH_INFO: `/v3/projects/ca31703335ed407a968681a602c24117` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:08.944542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca31703335ed407a968681a602c24117)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:08.946471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:08.965486 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:09.156709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.158337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.168633 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca31703335ed407a968681a602c24117', 'request_id': 'req-737b1c15-c732-4b79-b5cc-104b3c286bfd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.176496 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca31703335ed407a968681a602c24117', 'request_id': 'req-737b1c15-c732-4b79-b5cc-104b3c286bfd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.177765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9967eaf-e1f6-4f82-8e62-8ce44abff31c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.182166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:09.236370 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.237243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.237530 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.237827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] PATH_INFO: `/v3/projects/d00edd8bebc34098919b25aafca8974b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.239906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00edd8bebc34098919b25aafca8974b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.242106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.274948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca31703335ed407a968681a602c24117', 'request_id': 'req-737b1c15-c732-4b79-b5cc-104b3c286bfd'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.286006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] The token cache is being invalidate because project ca31703335ed407a968681a602c24117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:09.286294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.286916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-737b1c15-c732-4b79-b5cc-104b3c286bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.288481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1289/2585] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:08 2022] DELETE /identity/v3/projects/ca31703335ed407a968681a602c24117 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:09.293316 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40bbb-e7ad-4e78-a8c0-a3d582331406 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.293611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40bbb-e7ad-4e78-a8c0-a3d582331406 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.293894 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40bbb-e7ad-4e78-a8c0-a3d582331406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.312138 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8fc40bbb-e7ad-4e78-a8c0-a3d582331406 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:09.313410 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8fc40bbb-e7ad-4e78-a8c0-a3d582331406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.322015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1290/2586] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:09.329842 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.330167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.330477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.350492 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:09.351690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.359935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1291/2587] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:09.364846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.373981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54d38d-fd3f-4ef1-b4b2-15643bc22f95 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.380781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00edd8bebc34098919b25aafca8974b', 'request_id': 'req-67a86885-155d-47d9-80c0-485e92604b4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.386925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00edd8bebc34098919b25aafca8974b', 'request_id': 'req-67a86885-155d-47d9-80c0-485e92604b4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.430637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.431864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.432212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.432465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.433212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:09.433509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.435152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.449077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1292/2588] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:32:09.453507 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.463377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e5d08-35ba-4065-a86c-9354d097772d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.467982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.468656 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.468944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.469226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.469914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.471813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.566543 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00edd8bebc34098919b25aafca8974b', 'request_id': 'req-67a86885-155d-47d9-80c0-485e92604b4a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.577555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] The token cache is being invalidate because project d00edd8bebc34098919b25aafca8974b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:09.577844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.579897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-67a86885-155d-47d9-80c0-485e92604b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:09.580002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1297/2589] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:09 2022] DELETE /identity/v3/projects/d00edd8bebc34098919b25aafca8974b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:09.584936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000440c2-9c89-4cda-8463-90a45e679ebe None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.585223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000440c2-9c89-4cda-8463-90a45e679ebe None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.585500 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000440c2-9c89-4cda-8463-90a45e679ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.603896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-000440c2-9c89-4cda-8463-90a45e679ebe None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:09.605206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-000440c2-9c89-4cda-8463-90a45e679ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.613343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1298/2590] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:09.623226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.623517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.623798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.641488 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:09.642408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1293/2591] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:32:09.643432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.646827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.648519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:09.708450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1299/2592] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:09.709452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.714428 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb2be-c48e-4609-b03c-5d5c5bb27972 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.718343 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5ac21-e67c-40a0-a468-df6be624ff9b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.719350 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.720066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.720402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.720684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.721377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.723783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.723946 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.724466 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.724755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.725032 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.725772 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:09.726086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.727686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.739295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1300/2593] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:32:09.743864 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.753571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb942b57-3724-4fcf-bb30-c2d42b714da4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.758259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.758908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.759180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.759464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.760153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.761059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:09.763438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.858667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1294/2594] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:32:09.862969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.873180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843944d2-a2eb-4937-923d-a8d8631a9e3f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.877832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.878479 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.878768 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.879137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.880006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.881652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.891574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1295/2595] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:32:09.902310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:09.902310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1301/2596] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:32:09.905275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.906175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:09.916117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d904914-5ab9-4a05-bbe6-80f33def9de6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.969450 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b496835-e948-46c3-87d4-55a33411860c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:09.975736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.976355 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:09.976529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.976787 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.977025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:09.977125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.977314 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:09.977616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:09.977747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.978324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:09.979753 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.979919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:09.990417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1296/2597] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:09.994750 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.004883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694895e-8631-40e6-9bf3-87cc7e92d411 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.009581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.010456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.010741 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.011022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] PATH_INFO: `/v3/projects/246cc971eb1a4882868c39c39d27e3f5/users/38c44ad4254a46beba4a8a2ca5175f82/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.025939 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=246cc971eb1a4882868c39c39d27e3f5, user_id=38c44ad4254a46beba4a8a2ca5175f82, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.028667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8246e9a-6bca-4e10-b10a-a50e871a21ae admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.087212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1302/2598] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:09 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:32:10.091205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.101475 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5fb68-223b-45cd-b039-937ee51400eb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.105959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.106606 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.106900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.107184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.107957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.109590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.120841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1303/2599] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:32:10.121281 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1297/2600] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:09 2022] PUT /identity/v3/projects/246cc971eb1a4882868c39c39d27e3f5/users/38c44ad4254a46beba4a8a2ca5175f82/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:10.129595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.130399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.130814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.131090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.139843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf42ff3-9b0c-46af-b788-9d7b535dcf54 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.149163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:10.154548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:10.199047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.200435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.200733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.201020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.203051 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.203409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.215811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1304/2601] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:32:10.216850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1298/2602] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:10.220902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.226194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.227602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:10.231414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6ec26-a589-407c-a26b-2f640c5eddff admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.236720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.237585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.237880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.238172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] PATH_INFO: `/v3/projects/148d1010ef2845eb834df707e583a95c/users/03a7e1fe9cf94ed4abbafe820ef1f276/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.254464 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=148d1010ef2845eb834df707e583a95c, user_id=03a7e1fe9cf94ed4abbafe820ef1f276, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.257320 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.288934 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c588e8-ce55-4c2d-b3d7-dee9efe7d071 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.292719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.293376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.293669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.293956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.303275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.304785 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8397a6-77de-4016-bdf9-0038132689fd service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.308792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:10.314834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1299/2603] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:32:10.338837 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1305/2604] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:10 2022] PUT /identity/v3/projects/148d1010ef2845eb834df707e583a95c/users/03a7e1fe9cf94ed4abbafe820ef1f276/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:10.348967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.349262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.349547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.377213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] MFA Rules not processed for user `03a7e1fe9cf94ed4abbafe820ef1f276`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:10.378472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:10.448813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1300/2605] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:10.461356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.463788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:10.517063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:10.542901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e0b46-50d3-4216-ac40-d4eec1cbd7df admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.548489 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.549148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.549436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.549720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.558071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.559658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121173fb-3211-4bd8-8dfa-efcb0a37fb86 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:10.569046 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1306/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:10.813621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:10.902413 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650943cf-7c74-46aa-b61f-16e1273f8f55 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:10.908059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:10.909219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:10.909513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:10.909792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] PATH_INFO: `/v3/users/03a7e1fe9cf94ed4abbafe820ef1f276` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:10.912347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a7e1fe9cf94ed4abbafe820ef1f276)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:10.914552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:11.107053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a7e1fe9cf94ed4abbafe820ef1f276'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:11.107908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a7e1fe9cf94ed4abbafe820ef1f276'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:11.121300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a7e1fe9cf94ed4abbafe820ef1f276'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:11.128657 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a7e1fe9cf94ed4abbafe820ef1f276'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:11.235172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a7e1fe9cf94ed4abbafe820ef1f276'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:11.242519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1301/2607] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:10 2022] DELETE /identity/v3/users/03a7e1fe9cf94ed4abbafe820ef1f276 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:11.247203 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3183f-6451-4920-a24a-db4d326d61e3 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:11.247666 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3183f-6451-4920-a24a-db4d326d61e3 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:11.247763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3183f-6451-4920-a24a-db4d326d61e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:11.266373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-89d3183f-6451-4920-a24a-db4d326d61e3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:11.267653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-89d3183f-6451-4920-a24a-db4d326d61e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:11.329172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1307/2608] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:11.340332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:11.341772 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:11.352253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:11.401669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab92c0e-be36-4a11-bddf-522ad3cf07ee admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:11.405388 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:11.412526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:11.412526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:11.412526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:11.415744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:11.417242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:11.427673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1302/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:11.671178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:11.671648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:11.672006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:11.700697 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] MFA Rules not processed for user `38c44ad4254a46beba4a8a2ca5175f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:11.701958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:11.786517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1308/2610] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:11.797278 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:11.798671 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:11.855012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f69692-386c-480a-9439-2c80332bc703 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:11.858645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:11.859292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:11.859576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:11.859855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:11.868940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:11.878869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a31857-6243-4ea8-afb0-683f42ba3ef0 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:11.879945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1303/2611] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:11.916391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:11.917831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:11.932253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f00a5-883a-4ff2-9b80-50b8c4506d6e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:11.937121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:11.938072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:11.938357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:11.938635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] PATH_INFO: `/v3/projects/148d1010ef2845eb834df707e583a95c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:11.939868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=148d1010ef2845eb834df707e583a95c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:11.942405 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.081955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '148d1010ef2845eb834df707e583a95c', 'request_id': 'req-9aaf1187-f382-42c2-b3b4-3f84934d6c67'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:12.088249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '148d1010ef2845eb834df707e583a95c', 'request_id': 'req-9aaf1187-f382-42c2-b3b4-3f84934d6c67'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:12.090902 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:12.172001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '148d1010ef2845eb834df707e583a95c', 'request_id': 'req-9aaf1187-f382-42c2-b3b4-3f84934d6c67'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:12.183888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] The token cache is being invalidate because project 148d1010ef2845eb834df707e583a95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:12.184174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:12.184856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-9aaf1187-f382-42c2-b3b4-3f84934d6c67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:12.186411 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1309/2612] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:11 2022] DELETE /identity/v3/projects/148d1010ef2845eb834df707e583a95c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:12.191417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.192473 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.192737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.211323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:12.212638 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:12.273648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1304/2613] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:12.282925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.283376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.283486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.303234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:12.304477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:12.312723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1310/2614] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:12.317588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.327539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b5bbe-096a-4958-b0f3-1ec599366e97 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.332346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.333035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.333354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.333662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.334406 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:12.334718 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.336390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.348924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1305/2615] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:32:12.353164 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.362451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3c719-26b9-4aeb-a4da-610bc19ca6db None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.368171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.368866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.369158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.369435 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.370243 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.372153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.471015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1311/2616] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:12.475562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.486365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600775d6-1341-44e4-a955-cb2c74081fe9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.525027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:12.543680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.544382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.544669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.544945 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.545667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.547705 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.664530 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1306/2617] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:12.668493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.680006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74a078-45b1-448b-92fd-67827e2dc0a5 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.684786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.689015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.689015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.689015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.689015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.689015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.699431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1312/2618] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:12.703700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.714343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af81517-ec22-456d-afd3-4e63a4652797 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.719844 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.720474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.720775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.726797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.726797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.726797 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e64d2-9080-46ce-a4bb-aa5204c5441f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.735305 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1307/2619] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:12.739305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:12.750566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a4f91-1bea-4090-bc5d-3cdd5a194f6a admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:12.755358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:12.756296 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.756650 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.757110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] PATH_INFO: `/v3/projects/28bc9ed6847c404fa37be757fdc44ff1/users/3ec9cf6106804b388ef2129fb93ed27c/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.772936 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bc9ed6847c404fa37be757fdc44ff1, user_id=3ec9cf6106804b388ef2129fb93ed27c, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:12.775591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:12.870836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1313/2620] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:12 2022] PUT /identity/v3/projects/28bc9ed6847c404fa37be757fdc44ff1/users/3ec9cf6106804b388ef2129fb93ed27c/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:12.881951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:12.882251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:12.882534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:12.909135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] MFA Rules not processed for user `3ec9cf6106804b388ef2129fb93ed27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:12.910341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:12.976835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1308/2621] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:12.981251 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.044034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55f5ea-2372-4c26-8c16-3cc4be4ec9ec admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.048700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.049365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.049656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.049942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.050661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.053172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.144703 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1314/2622] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:12 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:13.149326 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.160226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc443c-a91b-41e1-8f7d-3dd64a22272f None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.220250 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.220968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.221234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.221519 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.222213 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.224169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.326144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1309/2623] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:13 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:13.330109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.340669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649078f-99cc-4bbb-9b98-fe912ebdc600 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.345452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.346105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.346398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.346679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.347457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.349873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.360916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1315/2624] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:13.366093 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.376375 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bd3a9-560e-42a2-b5c4-e8ffb6b04526 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.381332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.382239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.382555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.382812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] PATH_INFO: `/v3/projects/5e53b251295d4caaaec0677986a069c4/users/098805c118524427b326b22ad1148e5b/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.398518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e53b251295d4caaaec0677986a069c4, user_id=098805c118524427b326b22ad1148e5b, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.401283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.481339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1310/2625] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:13 2022] PUT /identity/v3/projects/5e53b251295d4caaaec0677986a069c4/users/098805c118524427b326b22ad1148e5b/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:13.485283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.495741 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ceee-06fa-40cd-b424-29aea3eddef8 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.545151 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:13.551901 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.552601 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.552887 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.553200 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.556414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.556414 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80436d-532b-41a2-932f-175590f027b3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.565558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1316/2626] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:13.569963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.584943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a76f5-3b9e-42a4-82b1-07b8e2dec950 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.587432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.588539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.588824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.589099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] PATH_INFO: `/v3/projects/5e53b251295d4caaaec0677986a069c4/users/098805c118524427b326b22ad1148e5b/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.591190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e53b251295d4caaaec0677986a069c4, user_id=098805c118524427b326b22ad1148e5b, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.594084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270803a8-1993-453f-ba95-540e9da51184 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.601041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:13.605619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1311/2627] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:13 2022] PUT /identity/v3/projects/5e53b251295d4caaaec0677986a069c4/users/098805c118524427b326b22ad1148e5b/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:13.616254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.616631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.616926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.642925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] MFA Rules not processed for user `098805c118524427b326b22ad1148e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:13.644265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:13.699955 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.700383 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.700597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.717945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1317/2628] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:13.719299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] MFA Rules not processed for user `38c44ad4254a46beba4a8a2ca5175f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:13.720548 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:13.729656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.731072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:13.778233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1312/2629] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:13.791843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278342-88f9-4dfd-a17b-89743032b740 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.795567 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.796228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.796555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.796839 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:13.806802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:13.808644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:13.820019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1318/2630] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:32:13.920509 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:13.931235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcb65c-eef8-4b23-81a7-d1f45f6ec611 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:13.935160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:13.935857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:13.936137 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:13.936449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:14.010095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:14.011807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a60aa-082d-425a-a008-f9a8d2df6b72 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:14.022733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1313/2631] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:17.249657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:17.318206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c72b-f56f-4866-9de2-4ba77c226952 service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:17.324456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:17.325358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:17.325649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:17.326555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] PATH_INFO: `/v3/users/3ec9cf6106804b388ef2129fb93ed27c` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:17.327358 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec9cf6106804b388ef2129fb93ed27c)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:17.329729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:17.356355 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:17.648944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec9cf6106804b388ef2129fb93ed27c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:17.656620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec9cf6106804b388ef2129fb93ed27c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:17.668596 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec9cf6106804b388ef2129fb93ed27c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:17.669281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec9cf6106804b388ef2129fb93ed27c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:17.669897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec9cf6106804b388ef2129fb93ed27c'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:17.807007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1319/2632] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:17 2022] DELETE /identity/v3/users/3ec9cf6106804b388ef2129fb93ed27c => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:17.811664 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:17.812200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:17.812354 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:17.830970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:17.832202 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:17.901644 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1314/2633] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:17.915193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:17.915193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:17.977360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8a09d-affd-4235-b754-02dedbdd21c3 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:17.981545 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:17.987064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:17.987064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:17.987064 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:17.993985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:17.996807 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:18.007232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1320/2634] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:32:18.539072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:18.541062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:18.553980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560bb4-e448-47e1-86b8-2070551a9c05 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:18.559998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:18.561356 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:18.562455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:18.562942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] PATH_INFO: `/v3/projects/28bc9ed6847c404fa37be757fdc44ff1` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:18.564857 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bc9ed6847c404fa37be757fdc44ff1)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:18.566821 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:18.717799 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:18.721889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bc9ed6847c404fa37be757fdc44ff1', 'request_id': 'req-a0b1c372-5a76-4114-8e9c-0a43e391ce68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:18.728654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bc9ed6847c404fa37be757fdc44ff1', 'request_id': 'req-a0b1c372-5a76-4114-8e9c-0a43e391ce68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:18.811829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bc9ed6847c404fa37be757fdc44ff1', 'request_id': 'req-a0b1c372-5a76-4114-8e9c-0a43e391ce68'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:18.827023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] The token cache is being invalidate because project 28bc9ed6847c404fa37be757fdc44ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:18.827023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:18.827023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:18.828512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1315/2635] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:18 2022] DELETE /identity/v3/projects/28bc9ed6847c404fa37be757fdc44ff1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:18.832300 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:18.843552 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecb230-0594-49fc-bfd3-54841161dd5c service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:18.908140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:18.909054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:18.909427 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:18.909712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] PATH_INFO: `/v3/users/098805c118524427b326b22ad1148e5b` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:18.910942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=098805c118524427b326b22ad1148e5b)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:18.913074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:19.178389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:19.181245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '098805c118524427b326b22ad1148e5b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:19.188688 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '098805c118524427b326b22ad1148e5b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:19.200931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '098805c118524427b326b22ad1148e5b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:19.201662 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '098805c118524427b326b22ad1148e5b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:19.202277 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1faf78c6-0a35-430b-8607-5f2f0fdd5b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '098805c118524427b326b22ad1148e5b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:19.345663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1321/2636] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:18 2022] DELETE /identity/v3/users/098805c118524427b326b22ad1148e5b => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:19.790560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:19.792392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:19.807865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1c372-5a76-4114-8e9c-0a43e391ce68 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:19.873402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:19.874334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:19.874627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:19.874910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] PATH_INFO: `/v3/projects/5e53b251295d4caaaec0677986a069c4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:19.876190 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e53b251295d4caaaec0677986a069c4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:19.878169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:19.994877 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e53b251295d4caaaec0677986a069c4', 'request_id': 'req-fb25da48-19d9-4863-9a1c-4fa02532f49d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:20.001690 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e53b251295d4caaaec0677986a069c4', 'request_id': 'req-fb25da48-19d9-4863-9a1c-4fa02532f49d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:20.109025 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e53b251295d4caaaec0677986a069c4', 'request_id': 'req-fb25da48-19d9-4863-9a1c-4fa02532f49d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:20.120615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] The token cache is being invalidate because project 5e53b251295d4caaaec0677986a069c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:20.120907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:20.121535 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-fb25da48-19d9-4863-9a1c-4fa02532f49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:20.123067 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1316/2637] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:19 2022] DELETE /identity/v3/projects/5e53b251295d4caaaec0677986a069c4 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:20.128020 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.128541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.128541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.146826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:20.148057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:20.208538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1322/2638] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:20.218160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.218458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.218745 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.238450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:20.239756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:20.249347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1317/2639] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:20.253747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.264354 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb256-a761-4dc9-8680-a64f79013218 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.269233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.269934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.270223 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.270502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.271236 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:20.271547 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.273310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.285788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1323/2640] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:32:20.290159 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.299684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1019752-5077-40d9-9a93-bab1a82846c2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.304593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.305254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.306078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.306400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.307276 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.309271 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.391749 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1318/2641] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:20.395817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.406316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ff3d-8904-4430-afe1-ee0e54a1f1ea admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.465494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.466180 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.466479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.466756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.467471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.469448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.572679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1324/2642] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:20.576890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.587275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdbe81-eeb4-44f2-9522-06e7055474b7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.592872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.592872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.592872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.593215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.593958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.595598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.606963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1319/2643] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:20.610982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.622309 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8210a14-e735-423c-9b49-144725a58107 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.626928 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.627592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.627881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.628166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.628941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.630592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583727c3-e874-44df-85ff-ec19eb82815c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.640709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1325/2644] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:20.644858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.654992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90809f25-7a18-42b3-abea-7aadefdb1d19 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.659660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.660560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.660845 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.661122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] PATH_INFO: `/v3/projects/e861f88b766247c6a1ae5187afd4e02f/users/956abb20e0384d69ad98749f7b97734f/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.676411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e861f88b766247c6a1ae5187afd4e02f, user_id=956abb20e0384d69ad98749f7b97734f, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.679060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:20.681980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:20.760698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1320/2645] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:20 2022] PUT /identity/v3/projects/e861f88b766247c6a1ae5187afd4e02f/users/956abb20e0384d69ad98749f7b97734f/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:20.770886 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.776273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.776273 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.799333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] MFA Rules not processed for user `956abb20e0384d69ad98749f7b97734f`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:20.800610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:20.815022 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:20.867014 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1326/2646] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:20.871697 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:20.937085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fb48a-ed5e-4bd5-ab6c-973ae8e9677b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:20.941960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:20.953654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:20.953654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:20.953654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:20.953654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:20.953654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.060018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1321/2647] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:20 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:21.064188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.076952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c016ac-b654-4c23-a2c0-39242f10e35c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.134035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.134867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.135146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.135430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.136139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.138097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.235502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1327/2648] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:21 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:21.239379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.250584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc508eb8-3bd6-4726-87c6-8a7518aef28f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.255251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.255917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.256205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.256552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.257315 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.259649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.270235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1322/2649] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:21.274229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.285023 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc5bf-a0b9-48b4-a75b-91c1ef8a4283 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.289595 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.290350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.290642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.290972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.291707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.293339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f800b-74bd-43c6-9d98-7e3b00e21242 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.303765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1328/2650] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:32:21.307891 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.318869 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e431f-f12e-4e3e-9540-b356ded75239 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.324269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.325214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.325498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.325776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] PATH_INFO: `/v3/projects/d3ddf3b938b8474fbeccde09a7032972/users/93c85f93b12e47e1b23be9be0d328244/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.341497 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ddf3b938b8474fbeccde09a7032972, user_id=93c85f93b12e47e1b23be9be0d328244, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.344158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.442063 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1323/2651] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:21 2022] PUT /identity/v3/projects/d3ddf3b938b8474fbeccde09a7032972/users/93c85f93b12e47e1b23be9be0d328244/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:21.453863 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.454166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.454454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.483509 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] MFA Rules not processed for user `93c85f93b12e47e1b23be9be0d328244`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:21.484830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:21.553361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1329/2652] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:21.570928 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.633443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c242-8b80-49a7-b398-1a133161fafb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.640157 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.640867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.641155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.641440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.642134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.644060 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.751921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1324/2653] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:21 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:21.756187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.767656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f99ff-db06-4f77-bcb5-4fd2ce0e2d1b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.787094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:21.823820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.824557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.825193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.825485 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.826263 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.828231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.955791 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1330/2654] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:21 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:21.959834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:21.970727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617662a8-4972-4b74-9b23-4dacd3cbd9d1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:21.975205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:21.975980 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:21.976266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:21.976586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:21.977307 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:21.979713 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:21.990796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1325/2655] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:21.995966 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:22.006249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1f4f0-fd75-4a77-b8dd-b7a3306e6203 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:22.011090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:22.012016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.012383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.012604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] PATH_INFO: `/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029/users/90b9c04b05c1463ba1523b54f2c8def4/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.027794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2d4430bc2048dd9ef5112c7ce69029, user_id=90b9c04b05c1463ba1523b54f2c8def4, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:22.030549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:22.160888 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1331/2656] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:21 2022] PUT /identity/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029/users/90b9c04b05c1463ba1523b54f2c8def4/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:22.165404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:22.175432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e68e7-1f23-4901-a7af-1eb32a18902b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:22.188135 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:22.230846 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:22.231526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.231812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.232119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.232879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:22.234572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae270c8-fcd4-478f-9707-7e3b4e681046 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:22.245536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1326/2657] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:22.249722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:22.260649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722adf0-bad5-462a-9006-c8f5a8292d99 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:22.265468 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:22.269831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.269831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.269831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] PATH_INFO: `/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029/users/90b9c04b05c1463ba1523b54f2c8def4/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.269831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2d4430bc2048dd9ef5112c7ce69029, user_id=90b9c04b05c1463ba1523b54f2c8def4, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:22.271979 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:22.283230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1332/2658] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:22 2022] PUT /identity/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029/users/90b9c04b05c1463ba1523b54f2c8def4/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:22.293871 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.294172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.294462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.323777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] MFA Rules not processed for user `90b9c04b05c1463ba1523b54f2c8def4`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:22.325096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:22.396853 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1327/2659] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:22.408653 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:22.410588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:22.473073 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461432a6-f358-4bcc-a0ca-acc06985cac6 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:22.476987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:22.480019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.480019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.480019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.488340 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:22.489788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:22.500081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1333/2660] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:22.626077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:22.627571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:22.640838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be5435-cd16-46e2-8156-a816762ee774 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:22.644580 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:22.645258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:22.645540 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:22.645818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:22.706691 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:22.708201 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faece0b-8d67-41ec-bcde-e16db4b011eb service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:22.717834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1328/2661] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:26.895244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:26.958518 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414864b4-b6a6-4e2b-8608-a06ae07d094e service cinder] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:26.963153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:26.964075 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:26.964425 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:26.964712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] PATH_INFO: `/v3/users/956abb20e0384d69ad98749f7b97734f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:26.965924 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956abb20e0384d69ad98749f7b97734f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:26.967990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:26.989495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:27.294942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956abb20e0384d69ad98749f7b97734f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:27.302109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956abb20e0384d69ad98749f7b97734f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:27.316004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956abb20e0384d69ad98749f7b97734f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:27.316751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956abb20e0384d69ad98749f7b97734f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:27.317357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956abb20e0384d69ad98749f7b97734f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:27.422312 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1334/2662] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:26 2022] DELETE /identity/v3/users/956abb20e0384d69ad98749f7b97734f => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:27.423820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:27.424269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:27.424565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:27.444066 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:27.445359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:27.482789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=100 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:27.514403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1329/2663] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:27.521259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:27.522644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:27.582254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1bf63-41ad-4eb4-83d6-77fc6a73ea58 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:27.586724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:27.587377 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:27.587686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:27.587951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:27.597938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:27.599379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:27.609893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1335/2664] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:28.122185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:28.123650 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:28.136155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528852ba-cf24-4a08-b250-20f116ee8c0c None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:28.140865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:28.141740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:28.142026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:28.142308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] PATH_INFO: `/v3/projects/e861f88b766247c6a1ae5187afd4e02f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:28.143500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e861f88b766247c6a1ae5187afd4e02f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:28.145489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:28.299782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e861f88b766247c6a1ae5187afd4e02f', 'request_id': 'req-817effb4-d7f0-403f-9b56-6a748adbda06'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.306194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e861f88b766247c6a1ae5187afd4e02f', 'request_id': 'req-817effb4-d7f0-403f-9b56-6a748adbda06'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.452351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e861f88b766247c6a1ae5187afd4e02f', 'request_id': 'req-817effb4-d7f0-403f-9b56-6a748adbda06'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.465528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] The token cache is being invalidate because project e861f88b766247c6a1ae5187afd4e02f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:28.465818 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.466481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.468114 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1330/2665] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:28 2022] DELETE /identity/v3/projects/e861f88b766247c6a1ae5187afd4e02f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:28.472121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:28.482467 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185349b8-8ca2-426d-a5c2-94bcdb9d5791 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:28.541648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:28.542752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:28.543044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:28.543328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] PATH_INFO: `/v3/users/93c85f93b12e47e1b23be9be0d328244` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:28.544566 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c85f93b12e47e1b23be9be0d328244)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:28.546496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:28.823696 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:28.826338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c85f93b12e47e1b23be9be0d328244'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.833600 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c85f93b12e47e1b23be9be0d328244'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.848257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c85f93b12e47e1b23be9be0d328244'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.849066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c85f93b12e47e1b23be9be0d328244'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.849676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c85f93b12e47e1b23be9be0d328244'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:28.932193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1336/2666] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:28 2022] DELETE /identity/v3/users/93c85f93b12e47e1b23be9be0d328244 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:29.397191 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:29.402027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:29.416179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817effb4-d7f0-403f-9b56-6a748adbda06 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:29.481727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:29.482594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:29.482883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:29.483176 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] PATH_INFO: `/v3/projects/d3ddf3b938b8474fbeccde09a7032972` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:29.484438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ddf3b938b8474fbeccde09a7032972)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:29.486398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:29.655333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ddf3b938b8474fbeccde09a7032972', 'request_id': 'req-2ef01d79-d0a8-4c42-a706-c84992fc68b5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:29.662050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ddf3b938b8474fbeccde09a7032972', 'request_id': 'req-2ef01d79-d0a8-4c42-a706-c84992fc68b5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:29.749151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ddf3b938b8474fbeccde09a7032972', 'request_id': 'req-2ef01d79-d0a8-4c42-a706-c84992fc68b5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:29.751639 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:29.762884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] The token cache is being invalidate because project d3ddf3b938b8474fbeccde09a7032972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:29.763258 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:29.764932 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:29.766786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1331/2667] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:29 2022] DELETE /identity/v3/projects/d3ddf3b938b8474fbeccde09a7032972 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:32:29.770816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:29.772211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:29.785962 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce14920-efcb-4caa-b175-6dae9416c9f4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:29.848896 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:29.849792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:29.850087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:29.850395 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] PATH_INFO: `/v3/users/90b9c04b05c1463ba1523b54f2c8def4` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:29.851652 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b9c04b05c1463ba1523b54f2c8def4)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:29.853641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:30.217374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b9c04b05c1463ba1523b54f2c8def4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.224700 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b9c04b05c1463ba1523b54f2c8def4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.243743 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b9c04b05c1463ba1523b54f2c8def4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.244546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b9c04b05c1463ba1523b54f2c8def4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.245237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4c5b98f8-14e3-4cf0-8606-5367d2ad7284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b9c04b05c1463ba1523b54f2c8def4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.310177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1337/2668] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:29 2022] DELETE /identity/v3/users/90b9c04b05c1463ba1523b54f2c8def4 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:30.745399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:30.746943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:30.764992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef01d79-d0a8-4c42-a706-c84992fc68b5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:30.831295 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:30.832164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:30.832587 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:30.832876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] PATH_INFO: `/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:30.834069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2d4430bc2048dd9ef5112c7ce69029)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:30.836199 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:30.939765 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2d4430bc2048dd9ef5112c7ce69029', 'request_id': 'req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:30.946083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2d4430bc2048dd9ef5112c7ce69029', 'request_id': 'req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:31.034601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2d4430bc2048dd9ef5112c7ce69029', 'request_id': 'req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:31.046409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] The token cache is being invalidate because project fc2d4430bc2048dd9ef5112c7ce69029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:31.046695 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:31.047324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-7fe4dcdf-321e-4b84-ae5f-b02bf34e2859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:31.048929 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1332/2669] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:30 2022] DELETE /identity/v3/projects/fc2d4430bc2048dd9ef5112c7ce69029 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:31.055079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.055599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.055726 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.074755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:31.076016 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:31.111333 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:31.137734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1338/2670] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:31.146545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.146838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.147122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.165868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:31.167088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:31.176669 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1333/2671] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:31.181109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.192002 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832685a7-21cc-428f-a91f-9a2bd911f60e None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.196632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.197289 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.201473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.201473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.201473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:31.201473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.201473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.215728 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1339/2672] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:32:31.220678 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.233545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88dabc-3b79-4e7e-876d-6144fc351ff1 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.238287 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.238963 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.239249 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.239532 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.240238 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.242462 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.376948 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1334/2673] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:31 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:31.378265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.388517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aff10-d84c-4944-980b-a65e4d994b07 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.467960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.468683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.468976 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.469258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.469963 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.471905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.562847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1340/2674] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:31 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:31.566834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.577449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa9cd5-e971-450c-82e0-78feda3fadc1 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.582139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.582759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.583048 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.583330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.584096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.585778 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.597235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1335/2675] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:31.601234 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.614860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82601e2d-0b55-4da6-bc2a-6ac5d90dca07 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.619946 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.620686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.620981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.621326 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.622106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.624471 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34777f69-977a-4008-aafd-3bb499016386 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.635479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1341/2676] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:31.640431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.650987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7087ab-e097-4133-9995-bda45d2b0ca0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:31.656576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:31.657467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.657755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.658033 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] PATH_INFO: `/v3/projects/a74f6072298d45c891fa4aee7bc8f137/users/935293d9cfb34a2fbb46745dc2318625/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.673368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74f6072298d45c891fa4aee7bc8f137, user_id=935293d9cfb34a2fbb46745dc2318625, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:31.676078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:31.835943 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1336/2677] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:31 2022] PUT /identity/v3/projects/a74f6072298d45c891fa4aee7bc8f137/users/935293d9cfb34a2fbb46745dc2318625/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:31.852054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:31.857291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:31.857291 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:31.885676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] MFA Rules not processed for user `935293d9cfb34a2fbb46745dc2318625`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:31.887237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:31.967970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1342/2678] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:31.979160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:31.981418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:32.007090 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:32.053748 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac46b7-b45a-4902-99d0-851286f0dfea admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:32.060763 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:32.061739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:32.062081 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:32.062523 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:32.072673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:32.074308 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e715e60-a77a-495d-8ac2-523cdb81882c service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:32.087629 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1337/2679] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:32.115319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:32.164643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:32.202121 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4d5ac-14a7-4f5e-8630-3ec25ea14c4b None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:32.207980 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:32.208906 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:32.209337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:32.209620 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] PATH_INFO: `/v3/users/935293d9cfb34a2fbb46745dc2318625` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:32.211037 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935293d9cfb34a2fbb46745dc2318625)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:32.213128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:32.463476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935293d9cfb34a2fbb46745dc2318625'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:32.489644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935293d9cfb34a2fbb46745dc2318625'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:32.508750 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935293d9cfb34a2fbb46745dc2318625'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:32.509683 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935293d9cfb34a2fbb46745dc2318625'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:32.510314 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935293d9cfb34a2fbb46745dc2318625'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:32.588834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1343/2680] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:32 2022] DELETE /identity/v3/users/935293d9cfb34a2fbb46745dc2318625 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:32.597528 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:32.597824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:32.598108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:32.628547 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:32.629464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:32.709430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1338/2681] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:32.721319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:32.722934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:32.854044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551dcce-a8ac-4117-8f02-ca8cddadf761 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:32.859144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:32.860267 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:32.860686 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:32.861126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:32.875568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:32.877255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a065c-e037-4a43-b8f7-1452fe242d5e service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:32.889738 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1344/2682] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:33.322600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:33.324062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:33.337732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6dcb3-cf88-4991-8b2e-811edf21d997 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:33.344841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:33.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] PATH_INFO: `/v3/projects/a74f6072298d45c891fa4aee7bc8f137` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:33.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74f6072298d45c891fa4aee7bc8f137)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:33.353836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:33.470393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74f6072298d45c891fa4aee7bc8f137', 'request_id': 'req-b5c87fbc-42bc-4557-a863-2a61743a23fb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:33.476904 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74f6072298d45c891fa4aee7bc8f137', 'request_id': 'req-b5c87fbc-42bc-4557-a863-2a61743a23fb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:33.550736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74f6072298d45c891fa4aee7bc8f137', 'request_id': 'req-b5c87fbc-42bc-4557-a863-2a61743a23fb'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:33.565786 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] The token cache is being invalidate because project a74f6072298d45c891fa4aee7bc8f137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:33.566083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:33.566755 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-b5c87fbc-42bc-4557-a863-2a61743a23fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:33.568545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1339/2683] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:33 2022] DELETE /identity/v3/projects/a74f6072298d45c891fa4aee7bc8f137 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:33.573985 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.574532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.574532 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:33.597331 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:33.598667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:33.660536 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:33.676470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1345/2684] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:33.680525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.680826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.681115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:33.703951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:33.705319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:33.717896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1340/2685] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:33.724035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:33.749647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d35901-664e-459b-a034-11816ce9ffa4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:33.757328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:33.758084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.758383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.758669 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:33.759502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:33.759819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:33.761612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:33.780062 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1346/2686] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:32:33.781290 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:33.791465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44e341-d926-43ce-80aa-f296cb15f711 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:33.797207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:33.797899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.798193 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.798478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:33.799218 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:33.801265 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:33.916962 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1341/2687] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:33 2022] POST /identity/v3/projects => generated 389 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:33.921367 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:33.932000 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c822a4-7c8a-4862-8cf0-3f65048f1466 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:33.998189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:33.998993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:33.999125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:33.999378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.000098 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:34.002070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:34.093339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1347/2688] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:33 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:34.097408 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:34.109104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c9e1-9761-4a4d-8a92-6d21672fd79b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:34.113990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:34.114666 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:34.121321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:34.121321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.121321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:34.121321 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:34.129754 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1342/2689] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:34.134295 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:34.145745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525cab-046a-4631-800d-24719cd94a19 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:34.150554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:34.151216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:34.151503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:34.151790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.153695 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:34.154258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e97c7-bcc2-4b8f-bb9b-bb40981c804d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:34.165206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1348/2690] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:34.169622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:34.176824 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:34.180576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd3ac-d744-4b65-819d-f452114a8627 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:34.185335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:34.186761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:34.187235 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:34.187566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] PATH_INFO: `/v3/projects/eee96b97948644ea8ce97c0013789392/users/4500292902254aa78e745cba8fe04268/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.203959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee96b97948644ea8ce97c0013789392, user_id=4500292902254aa78e745cba8fe04268, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:34.206951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:34.286898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1343/2691] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:34 2022] PUT /identity/v3/projects/eee96b97948644ea8ce97c0013789392/users/4500292902254aa78e745cba8fe04268/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:34.297463 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:34.297758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:34.302993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.326640 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] MFA Rules not processed for user `4500292902254aa78e745cba8fe04268`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:34.328496 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:34.398004 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1349/2692] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:34.409101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:34.410801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:34.474467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d332eb-77fa-41c3-85a7-b602afe9c9bb admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:34.478303 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:34.478995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:34.479282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:34.479561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:34.488608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:34.490042 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:34.499940 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1344/2693] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:40.691501 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:40.693103 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:40.752858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cd4b1-bcdd-4680-a007-400668ebadc4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:40.756550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:40.757210 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:40.757498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:40.757859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:40.762982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:40.766746 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:40.827730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:40.829292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:40.840059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1350/2694] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:32:44.434584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:44.435994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:44.516447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4578b-e3c8-4357-9983-2beb57743bef service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:44.522346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:44.522995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:44.523280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:44.523559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:44.525361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:44.537484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:44.539078 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:44.553882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1345/2695] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:44.605368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:44.606851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:44.620195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bf91-6d4a-4bea-b62a-7a529191a105 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:44.624951 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:44.625826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:44.626109 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:44.626436 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] PATH_INFO: `/v3/users/d5148114d71b401ca20f988485fb6c88` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:44.628166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5148114d71b401ca20f988485fb6c88)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:44.630195 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:45.042448 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5148114d71b401ca20f988485fb6c88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.049826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5148114d71b401ca20f988485fb6c88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.061836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5148114d71b401ca20f988485fb6c88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.062522 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5148114d71b401ca20f988485fb6c88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.063130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-ffc56f7e-a536-4503-b24f-97b10e777c89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5148114d71b401ca20f988485fb6c88'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.189290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1351/2696] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:44 2022] DELETE /identity/v3/users/d5148114d71b401ca20f988485fb6c88 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:45.448400 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:45.460897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e8ca0-1cc2-44ea-92ee-5ed06cd97a20 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:45.522147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:45.523016 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:45.523309 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:45.523598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] PATH_INFO: `/v3/projects/775c525b714e4f13ac5f34314243fe93` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:45.524833 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] RBAC: Authorizing `identity:delete_project(project_id=775c525b714e4f13ac5f34314243fe93)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:45.526766 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:45.543899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:45.664214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775c525b714e4f13ac5f34314243fe93', 'request_id': 'req-2a669d3b-854d-4661-9a6b-bd73de24a85d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.670552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775c525b714e4f13ac5f34314243fe93', 'request_id': 'req-2a669d3b-854d-4661-9a6b-bd73de24a85d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.823109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775c525b714e4f13ac5f34314243fe93', 'request_id': 'req-2a669d3b-854d-4661-9a6b-bd73de24a85d'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.835379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] The token cache is being invalidate because project 775c525b714e4f13ac5f34314243fe93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:45.835667 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.836296 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-2a669d3b-854d-4661-9a6b-bd73de24a85d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:45.837861 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1346/2697] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:45 2022] DELETE /identity/v3/projects/775c525b714e4f13ac5f34314243fe93 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:45.843071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:45.843492 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:45.843658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:45.862814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:45.864024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:45.927598 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1352/2698] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:45.935571 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:45.945627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:45.945627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:45.955279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:45.957214 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:45.965477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1347/2699] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:45.969768 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:45.979829 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4177868-1c91-45ae-adbd-d3bc2b2a4f42 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:45.984594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:45.985237 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:45.985525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:45.985806 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:45.986551 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:45.986851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:45.988479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.000245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1353/2700] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:32:46.005123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.014649 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3666e45-6fa9-4ea1-a2e3-41fcacf5d8f8 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.019990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.020682 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.020970 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.021246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.026028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.026028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.151455 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1348/2701] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:46 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:46.155441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.161188 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:46.166751 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6de16d-eb93-4362-8c56-7554fc7615fc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.223389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.224070 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.224385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.224670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.225369 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.227290 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.326517 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1354/2702] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:46 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:46.330495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.341299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f39028-ca5f-4cff-aead-f73a4d3ce268 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.346305 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.352421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.352421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.352421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.352421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.352421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.360899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1349/2703] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:46.365631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.375770 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a13d1-bb27-4552-866d-f49954e90e78 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.381030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.381691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.381981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.382261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.382997 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.384632 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c9a1a-99e7-410f-a043-e1823684bb63 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.394392 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1355/2704] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:46.398534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.408883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80e0cb-cb03-48ea-98dc-376bed06ddc0 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.413572 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.414440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.414731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.415010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] PATH_INFO: `/v3/projects/cec39457db974eebbc0305cc555a64d4/users/4d41a6703cec435dbb7e687aa0088d05/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.429907 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec39457db974eebbc0305cc555a64d4, user_id=4d41a6703cec435dbb7e687aa0088d05, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.432622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.521359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1350/2705] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:46 2022] PUT /identity/v3/projects/cec39457db974eebbc0305cc555a64d4/users/4d41a6703cec435dbb7e687aa0088d05/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:46.526583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e454f7-642c-4f88-a253-900e8ed00c0a None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.526875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e454f7-642c-4f88-a253-900e8ed00c0a None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.527155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e454f7-642c-4f88-a253-900e8ed00c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.545657 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-71e454f7-642c-4f88-a253-900e8ed00c0a None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:46.546869 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-71e454f7-642c-4f88-a253-900e8ed00c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:46.615819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1356/2706] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:32:46.624343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:46.626104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:46.685793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1c77b-6fd0-43c4-ab51-d7bed21ca8fc admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:46.689461 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:46.690373 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:46.690660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:46.690944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:46.700186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:46.702133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:46.711719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1351/2707] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:52.333824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91d8b-e7c6-4921-8f52-c91d3e859e1e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:52.334795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91d8b-e7c6-4921-8f52-c91d3e859e1e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:52.336744 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b91d8b-e7c6-4921-8f52-c91d3e859e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:52.381431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-f4b91d8b-e7c6-4921-8f52-c91d3e859e1e None None] MFA Rules not processed for user `4d41a6703cec435dbb7e687aa0088d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:52.383820 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f4b91d8b-e7c6-4921-8f52-c91d3e859e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:52.466733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1357/2708] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:52.483637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:52.485608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:52.535513 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:52.574154 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9ff82-b658-4f74-a176-aa910d93bc5a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:52.580595 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:52.586072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:52.586072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:52.586072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:52.594500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:52.596129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:52.614454 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1352/2709] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:55.260387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0860-20bc-4c81-a9a4-18b84e2119e1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:55.262525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0860-20bc-4c81-a9a4-18b84e2119e1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:55.262525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0860-20bc-4c81-a9a4-18b84e2119e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:55.282174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ff1b0860-20bc-4c81-a9a4-18b84e2119e1 None None] MFA Rules not processed for user `4d41a6703cec435dbb7e687aa0088d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:55.284168 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ff1b0860-20bc-4c81-a9a4-18b84e2119e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:55.292831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1358/2710] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:55.955935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:55.957689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:55.972219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183229e-e92f-4b79-87bb-45221142a6b0 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:55.977215 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:55.978108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:55.978399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:55.978681 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] PATH_INFO: `/v3/users/4500292902254aa78e745cba8fe04268` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:55.979910 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4500292902254aa78e745cba8fe04268)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:55.982129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:56.176867 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4500292902254aa78e745cba8fe04268'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:56.178164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4500292902254aa78e745cba8fe04268'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:56.196510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4500292902254aa78e745cba8fe04268'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:56.203591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4500292902254aa78e745cba8fe04268'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:56.274771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4500292902254aa78e745cba8fe04268'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:56.281541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1353/2711] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:32:55 2022] DELETE /identity/v3/users/4500292902254aa78e745cba8fe04268 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:56.286338 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:56.286824 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:56.286949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:56.294060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:56.305802 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:56.307110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:56.380892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1359/2712] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:56.390835 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:56.392266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:56.456333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278414ee-b186-49c6-bf35-981970ac2332 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:56.460825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:56.461557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:56.461777 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:56.462061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:56.472796 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:56.474425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe751312-8fa8-45a0-8fd1-366da3698e5e service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:56.484134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1354/2713] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:32:56.950647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:56.952421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:56.968059 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec072f-07d6-46a2-912d-c06095fddee2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:56.974118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:56.975060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:56.975422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:56.975803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] PATH_INFO: `/v3/projects/eee96b97948644ea8ce97c0013789392` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:56.977090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee96b97948644ea8ce97c0013789392)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:56.979265 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.145136 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee96b97948644ea8ce97c0013789392', 'request_id': 'req-f96c02fc-5445-4854-9125-5ccc82d467f4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:57.151897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee96b97948644ea8ce97c0013789392', 'request_id': 'req-f96c02fc-5445-4854-9125-5ccc82d467f4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:57.257611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee96b97948644ea8ce97c0013789392', 'request_id': 'req-f96c02fc-5445-4854-9125-5ccc82d467f4'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:57.270211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] The token cache is being invalidate because project eee96b97948644ea8ce97c0013789392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:32:57.270211 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:57.270856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f96c02fc-5445-4854-9125-5ccc82d467f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:32:57.272549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1360/2714] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:32:56 2022] DELETE /identity/v3/projects/eee96b97948644ea8ce97c0013789392 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:57.279118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.279617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.279740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.297429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:57.298646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:57.351801 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:57.374051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1355/2715] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:57.383739 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.384253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.384493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.403185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:57.404449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:57.412591 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1361/2716] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:57.416999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.427611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c630-ea11-4eeb-aa2e-a08df29a1917 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.432959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.433625 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.433906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.434185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.434923 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:32:57.435233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.437013 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.450110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1356/2717] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:32:57.454515 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.464175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450bbe1-cc62-4187-abfc-3e1a4f795ebb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.468913 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.469571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.469860 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.470140 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.470883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.473054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.556678 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1362/2718] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:32:57 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:57.560982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.571731 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b185a5-6e9d-4e2e-9bdc-bc9b4cbd4a03 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.632236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.633077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.633502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.633823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.634527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.636531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.727536 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1357/2719] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:32:57 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:32:57.731507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.743173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03230c33-52ec-493f-a5c4-3e754201349d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.747826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.748514 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.748803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.749083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.749818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.751420 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.763088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1363/2720] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:57.766597 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.777115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c208b-2c78-4aed-849e-7d2427e5b6d9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.782039 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.782393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.782683 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.782968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.783702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.785969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd09de-1b04-491f-a1cc-fed7683d4016 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.795968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1358/2721] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:32:57.800845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:57.811544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17c86-3f46-405b-bdd1-9eac08b3b494 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:57.816944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:57.828107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.828107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.828107 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] PATH_INFO: `/v3/projects/447abd08211d49f9b503130ac5c380b7/users/7a647aab17cf4345800cd9072a7820c0/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.834537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] RBAC: Authorizing `identity:create_grant(project_id=447abd08211d49f9b503130ac5c380b7, user_id=7a647aab17cf4345800cd9072a7820c0, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:57.837242 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:57.932718 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1364/2722] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:32:57 2022] PUT /identity/v3/projects/447abd08211d49f9b503130ac5c380b7/users/7a647aab17cf4345800cd9072a7820c0/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:32:57.937887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cefbd7-c87c-46da-8ee7-205c0a9cb45e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:57.938181 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cefbd7-c87c-46da-8ee7-205c0a9cb45e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:57.938467 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cefbd7-c87c-46da-8ee7-205c0a9cb45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:57.957028 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-b7cefbd7-c87c-46da-8ee7-205c0a9cb45e None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:32:57.958253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b7cefbd7-c87c-46da-8ee7-205c0a9cb45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:58.021212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1359/2723] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:32:58.031026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:32:58.032808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:32:58.061274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:32:58.089079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b99f2d-43ce-4302-b9d4-ba4237fc954d admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:32:58.092731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:32:58.093379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:32:58.093658 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:32:58.093935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:32:58.103611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:32:58.105079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:32:58.115123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1365/2724] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:32:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:33:04.587136 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:04.587659 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:04.587840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:04.617889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] MFA Rules not processed for user `7a647aab17cf4345800cd9072a7820c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:33:04.619158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:04.659905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:04.693317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1360/2725] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:33:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:33:04.697879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:04.708706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bee20-9898-444f-8817-efe99d6efffd service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:04.713879 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:04.714634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:04.714945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:04.715213 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:04.715919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:04.717883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:04.883351 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1366/2726] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:33:04 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:33:04.887333 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:04.898987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0b2e1-ea09-48e5-97b6-e9d3cc3f099d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:04.985733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:04.986438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:04.986737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:04.987111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:04.988611 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:04.990739 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:05.121512 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1361/2727] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:33:04 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:33:05.125583 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:05.137730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866d7f7-0e8c-4965-90c0-8d395c05e0df admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:05.142641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:05.143385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:05.149185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:05.149185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:05.149185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:05.149185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:05.158826 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1367/2728] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:33:05.163471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:05.174495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec4cdc-05a5-452c-8a9a-00bebfefcd6f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:05.179884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:05.180839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:05.181128 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:05.181439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] PATH_INFO: `/v3/projects/e56d35c43e134bb7b29ea3e00d165cab/users/ebdeaae563b543a4b07eb7b246b888ea/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:05.198261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56d35c43e134bb7b29ea3e00d165cab, user_id=ebdeaae563b543a4b07eb7b246b888ea, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:05.201056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:05.269378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1362/2729] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:33:05 2022] PUT /identity/v3/projects/e56d35c43e134bb7b29ea3e00d165cab/users/ebdeaae563b543a4b07eb7b246b888ea/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:33:05.273554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:05.284851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738f3e18-029a-47dd-9d18-d3aa516beb34 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:05.343993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:05.344836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:05.345129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:05.345418 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:05.346232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:05.349233 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1b5f6-f501-40c9-a019-26028e44eaea admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:05.360675 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1368/2730] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:33:05.365688 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:05.377565 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8362-5510-4df9-ae20-65f54db5419d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:05.382672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:05.383573 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:05.383872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:05.384160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] PATH_INFO: `/v3/projects/e56d35c43e134bb7b29ea3e00d165cab/users/ebdeaae563b543a4b07eb7b246b888ea/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:05.386267 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56d35c43e134bb7b29ea3e00d165cab, user_id=ebdeaae563b543a4b07eb7b246b888ea, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:05.389073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:05.401005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1363/2731] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:33:05 2022] PUT /identity/v3/projects/e56d35c43e134bb7b29ea3e00d165cab/users/ebdeaae563b543a4b07eb7b246b888ea/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:33:12.164725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:12.165633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:12.165937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:12.232057 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] MFA Rules not processed for user `ebdeaae563b543a4b07eb7b246b888ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:33:12.233400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:12.313215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:12.342907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1369/2732] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:33:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:33:12.357421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:12.361422 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:12.477583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:12.531806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b85b2-8ebf-4bda-8d86-84bdb145d158 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:12.538419 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:12.539124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:12.539421 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:12.539740 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:12.600242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:12.601925 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a9a02-9fd3-4394-a890-c6715d287397 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:12.611058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1364/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:33:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:33:23.474499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=8 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:23.543703 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=45 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:24.976407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:24.977821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:25.077381 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780d688-dcb4-446c-ae1f-662d2c9957ee None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:25.083875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:25.084737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:25.085028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:25.085308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:25.087080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:25.101357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:25.103028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53d19b-da38-4794-af54-3e4668b72212 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:25.114055 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1370/2734] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:33:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:33:35.110361 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=41 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:37.082155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:37.082692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:37.082692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:37.101328 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] MFA Rules not processed for user `38c44ad4254a46beba4a8a2ca5175f82`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:33:37.102526 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:37.156725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1365/2735] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:33:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:33:37.764575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18599-03b5-4e0b-b06e-5e37409c0f88 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:37.765091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18599-03b5-4e0b-b06e-5e37409c0f88 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:37.765191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f18599-03b5-4e0b-b06e-5e37409c0f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:37.789763 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a3f18599-03b5-4e0b-b06e-5e37409c0f88 None None] MFA Rules not processed for user `7a647aab17cf4345800cd9072a7820c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:33:37.791018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a3f18599-03b5-4e0b-b06e-5e37409c0f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:37.799131 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1371/2736] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:33:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:33:47.149969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=25 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:47.783083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:33:55.684747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:33:55.686167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:33:55.764774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe47c1-0711-4ad1-82ad-4f9fe680e4a2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:33:55.768378 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:33:55.768999 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:33:55.769283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:33:55.769581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:33:55.778684 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:33:55.780121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:33:55.790210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1366/2737] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:33:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:34:05.785365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=35 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:34:10.053880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:34:10.054429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:34:10.054429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:34:10.072465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] MFA Rules not processed for user `4d41a6703cec435dbb7e687aa0088d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:34:10.073676 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:34:10.128592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1372/2738] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:34:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:34:20.121803 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=25 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:34:45.714485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:34:45.715966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:34:45.816609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fabe6-69da-4e22-b49c-29dacb0d09b9 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:34:45.821425 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:34:45.822291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:34:45.822579 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:34:45.822856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] PATH_INFO: `/v3/users/38c44ad4254a46beba4a8a2ca5175f82` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:34:45.824058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c44ad4254a46beba4a8a2ca5175f82)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:34:45.826024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:34:46.042554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c44ad4254a46beba4a8a2ca5175f82'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.043120 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c44ad4254a46beba4a8a2ca5175f82'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.055006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c44ad4254a46beba4a8a2ca5175f82'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.061977 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c44ad4254a46beba4a8a2ca5175f82'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.142236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c44ad4254a46beba4a8a2ca5175f82'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.152478 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1367/2739] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:34:45 2022] DELETE /identity/v3/users/38c44ad4254a46beba4a8a2ca5175f82 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:34:46.384396 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:34:46.395308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df879585-e4fa-4dac-a958-4f5b89d05778 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:34:46.455387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:34:46.456259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:34:46.456634 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:34:46.456922 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] PATH_INFO: `/v3/projects/246cc971eb1a4882868c39c39d27e3f5` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:34:46.458122 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=246cc971eb1a4882868c39c39d27e3f5)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:34:46.460085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:34:46.589128 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246cc971eb1a4882868c39c39d27e3f5', 'request_id': 'req-e2c35003-ebc7-43e0-89c2-633c964c8bbe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.595316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246cc971eb1a4882868c39c39d27e3f5', 'request_id': 'req-e2c35003-ebc7-43e0-89c2-633c964c8bbe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.702603 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246cc971eb1a4882868c39c39d27e3f5', 'request_id': 'req-e2c35003-ebc7-43e0-89c2-633c964c8bbe'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.714813 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] The token cache is being invalidate because project 246cc971eb1a4882868c39c39d27e3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:34:46.717310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.717310 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:34:46.717664 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1373/2740] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:34:46 2022] DELETE /identity/v3/projects/246cc971eb1a4882868c39c39d27e3f5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:34:56.145085 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=66,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:34:56.710876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:35:10.479233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:35:10.480767 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:10.560423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376efd7-2109-42b3-96b2-5d5fbacb7431 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:35:10.608924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:35:10.610234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:10.610474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:10.610784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:10.612626 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:10.683985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:35:10.685493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0baf8-ef85-4add-b605-f0abb95bb680 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:35:10.697158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1368/2741] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:35:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:35:13.791798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:35:13.793249 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:13.853398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c35003-ebc7-43e0-89c2-633c964c8bbe admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:35:13.857385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:35:13.858039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:13.858330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:13.858614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:13.860875 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:13.874958 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:35:13.876594 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:35:13.887609 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1374/2742] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:35:20.692324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=83 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:35:22.247268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:22.247889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:22.247889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:22.266306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] MFA Rules not processed for user `4d41a6703cec435dbb7e687aa0088d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:35:22.268151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:22.322349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1369/2743] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:35:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:35:22.626775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:35:22.635709 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c6de4-3b13-4be3-a41c-6be1e96ef5a5 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:35:22.639499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:35:22.640149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:22.640476 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:22.640758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:22.642782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:22.653229 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:35:22.654668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:35:22.663453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1375/2744] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:35:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:35:32.315429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=25 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:35:32.659828 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=32 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:35:34.572001 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:35:34.581878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffd82-0d29-4e2b-b0db-888bf72749df None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:35:34.586243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:35:34.586899 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:34.587185 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:34.587491 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:34.589379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:34.600242 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:35:34.601736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da31f-6966-4313-b60a-826b1c282dd9 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:35:34.610297 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1370/2745] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:35:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:35:44.606518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:35:59.148781 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:35:59.152815 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:59.174790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd4849-8d82-4e07-8127-36a2548c85db service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:35:59.182851 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:35:59.183831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:35:59.184339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:35:59.184836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:35:59.187573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:35:59.204254 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:35:59.210116 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:35:59.227232 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1376/2746] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:35:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:36:01.078634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:36:01.079288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:36:01.079288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:36:01.097057 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] MFA Rules not processed for user `7a647aab17cf4345800cd9072a7820c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:36:01.098277 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:01.154843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1371/2747] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:36:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:36:09.219181 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:36:11.147935 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=25 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:36:16.482575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:36:16.484283 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:16.498989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263223f0-a3df-48d6-957e-94a8308396ed service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:36:16.505118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:36:16.505774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:36:16.506069 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:36:16.506356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:36:16.508148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:16.522722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:36:16.526305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7632d14-9623-46e8-a9fd-37484e24ca25 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:36:16.536208 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1377/2748] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:36:20.983502 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:36:21.000392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe5fc4-92ad-4955-9e23-363112a07756 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:36:21.007702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:36:21.008521 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:36:21.008880 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:36:21.009234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:36:21.011177 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:21.025164 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:36:21.026901 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:36:21.039674 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1372/2749] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:36:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:36:26.530884 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=6 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:36:31.034143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=6 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:36:32.772457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:36:32.772957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:36:32.773091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:36:32.792322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] MFA Rules not processed for user `4d41a6703cec435dbb7e687aa0088d05`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:36:32.793538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:32.800615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1378/2750] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:36:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:36:34.598716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:36:34.600171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:34.676975 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a0cb3-9465-4b09-8321-a1d32c057261 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:36:34.682113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:36:34.682764 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:36:34.683052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:36:34.683337 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:36:34.685230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:36:34.696648 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:36:34.698301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:36:34.708109 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1373/2751] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:36:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:36:42.787041 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:36:44.703959 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=32 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:47.671384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:47.673006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:47.685607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dec4a-ff56-4298-956e-c2c1428fa91d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:47.689856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:47.690519 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:47.690808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:47.691093 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:47.693058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:47.761135 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:47.762593 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:47.772833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1379/2752] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:37:48.724956 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:48.737593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794ffee-029a-4d4e-9483-2fe3a93a06c1 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:48.743324 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:48.744072 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:48.744469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:48.744826 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:48.747119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:48.761522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:48.763262 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:48.774832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1374/2753] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:37:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:37:48.866175 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:48.867734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:48.881592 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53c95c-6d0d-4f30-b741-c831cd384023 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:48.886733 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:48.887849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:48.888171 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:48.888445 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] PATH_INFO: `/v3/users/4d41a6703cec435dbb7e687aa0088d05` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:48.889663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d41a6703cec435dbb7e687aa0088d05)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:48.891635 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:49.172794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d41a6703cec435dbb7e687aa0088d05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.180546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d41a6703cec435dbb7e687aa0088d05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.193409 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d41a6703cec435dbb7e687aa0088d05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.196495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d41a6703cec435dbb7e687aa0088d05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.196495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-c44d35c8-55d9-4c4c-830b-fefb674a88b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d41a6703cec435dbb7e687aa0088d05'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.315512 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1380/2754] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:37:48 2022] DELETE /identity/v3/users/4d41a6703cec435dbb7e687aa0088d05 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:49.571196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:49.583285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886df1b-08de-4396-89bb-81bae7478398 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:49.644291 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:49.645194 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:49.645485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:49.645774 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] PATH_INFO: `/v3/projects/cec39457db974eebbc0305cc555a64d4` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:49.646972 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec39457db974eebbc0305cc555a64d4)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:49.649007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:49.771061 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec39457db974eebbc0305cc555a64d4', 'request_id': 'req-543a4cdb-9669-41c2-a4da-b939e66cb358'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.777449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec39457db974eebbc0305cc555a64d4', 'request_id': 'req-543a4cdb-9669-41c2-a4da-b939e66cb358'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.858351 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec39457db974eebbc0305cc555a64d4', 'request_id': 'req-543a4cdb-9669-41c2-a4da-b939e66cb358'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.870168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] The token cache is being invalidate because project cec39457db974eebbc0305cc555a64d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:37:49.870460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.871145 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-543a4cdb-9669-41c2-a4da-b939e66cb358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:49.873017 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1375/2755] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:37:49 2022] DELETE /identity/v3/projects/cec39457db974eebbc0305cc555a64d4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:49.878790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:49.879255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:49.879360 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:49.938082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:49.940391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:49.972231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:50.000681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1381/2756] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:50.009449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.009736 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.010019 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.029083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:50.030294 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:50.038306 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1376/2757] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:50.042724 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.052894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9020de-e0cf-4ae9-a166-bb40988eec25 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.057692 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.058366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.058778 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.062130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.062130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:37:50.062130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.062130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.074649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1382/2758] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 18:37:50.078921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.088987 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af11b9-d9c8-477b-a3be-d035525f07e4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.093635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.094282 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.094569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.094856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.095556 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.097517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.209518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1377/2759] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:37:50 2022] POST /identity/v3/projects => generated 403 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:50.213944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.224538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9cd4ff-994d-40bc-bd6a-7425d35e203e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.281948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.282762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.283040 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.283441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.284157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.286129 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.399053 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1383/2760] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:37:50 2022] POST /identity/v3/users => generated 302 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:50.403221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.413601 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24950c-11f6-4939-bf5f-983b13b824b3 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.418125 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.418783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.419070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.419628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.420150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.421795 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.432219 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1378/2761] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:50.436118 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.447459 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e576-1419-4e5a-b90d-63c04ec9686c admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.452324 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.452996 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.453287 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.453605 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.454421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.456097 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bee8d-26e1-44d4-9fd9-f9c9cee94746 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.466742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1384/2762] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:37:50.470784 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.480832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3138476-4edc-4132-90b8-65d55dffc93f admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.485414 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.486885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.487173 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.487452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] PATH_INFO: `/v3/projects/a7147d7d71f34eb9bed7849d7b670d48/users/f6ef4f5010364f1481a71cb1ab18d4d5/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.503005 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7147d7d71f34eb9bed7849d7b670d48, user_id=f6ef4f5010364f1481a71cb1ab18d4d5, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.505725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.599593 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1379/2763] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:50 2022] PUT /identity/v3/projects/a7147d7d71f34eb9bed7849d7b670d48/users/f6ef4f5010364f1481a71cb1ab18d4d5/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:50.604855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5285f1-a4c8-4efd-a1a2-4ed784b875a7 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.605153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5285f1-a4c8-4efd-a1a2-4ed784b875a7 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.605430 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5285f1-a4c8-4efd-a1a2-4ed784b875a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.623894 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-3f5285f1-a4c8-4efd-a1a2-4ed784b875a7 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:50.625844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-3f5285f1-a4c8-4efd-a1a2-4ed784b875a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:50.685325 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1385/2764] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:50.694990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:50.696780 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:50.703339 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:50.755822 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cfb15-3d3b-4b25-b950-698495405253 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:50.759592 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:50.760248 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:50.760709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:50.761452 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:50.771409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:50.772916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:50.783893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1380/2765] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:37:52.264449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.265192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.265192 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.293431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] MFA Rules not processed for user `f6ef4f5010364f1481a71cb1ab18d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:52.294654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:52.328529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:52.364502 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1386/2766] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:52.369312 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.379539 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b58ec-d3c7-4d65-9206-e0bf19449c85 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.384716 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.385418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.385680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.385960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.386757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.388761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:52.545459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1381/2767] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:37:52 2022] POST /identity/v3/projects => generated 403 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:52.549870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.560655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb268155-5353-4222-bc7b-257339ec13e4 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.618776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.619449 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.619756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.620389 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.620747 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.622931 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:52.708347 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1387/2768] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:37:52 2022] POST /identity/v3/users => generated 308 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:52.712115 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.723121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627072df-231c-4936-91c0-e81747f50cde admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.727701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.728392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.728686 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.728968 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.729706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.731342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:52.743293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1382/2769] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:52.747558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.757633 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0632e1-5d00-49f3-a4b2-cf2d3effe758 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.762215 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.763868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.763868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.763868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] PATH_INFO: `/v3/projects/70c4035711fd4c1aba593dea5174ee92/users/6fa5255eb8e3471b82492b46ce940ea3/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.780221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c4035711fd4c1aba593dea5174ee92, user_id=6fa5255eb8e3471b82492b46ce940ea3, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.782949 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:52.873549 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1388/2770] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:52 2022] PUT /identity/v3/projects/70c4035711fd4c1aba593dea5174ee92/users/6fa5255eb8e3471b82492b46ce940ea3/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:52.881862 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.891858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8af5c-7681-443b-82ab-79938d2feabf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.950924 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.951602 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.951921 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.952211 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.952982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.954596 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e871c-94ab-41f9-b09d-0d0c8c728808 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:52.965263 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1383/2771] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:52.969454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:52.979849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b75b69-af8d-4857-a38b-edac6c191211 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:52.984558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:52.985446 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:52.985736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:52.986015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] PATH_INFO: `/v3/projects/70c4035711fd4c1aba593dea5174ee92/users/6fa5255eb8e3471b82492b46ce940ea3/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:52.988173 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c4035711fd4c1aba593dea5174ee92, user_id=6fa5255eb8e3471b82492b46ce940ea3, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:52.991076 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:53.005224 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1389/2772] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:52 2022] PUT /identity/v3/projects/70c4035711fd4c1aba593dea5174ee92/users/6fa5255eb8e3471b82492b46ce940ea3/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:54.294981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:54.295578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:54.295578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:54.325726 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] MFA Rules not processed for user `6fa5255eb8e3471b82492b46ce940ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:54.326967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:54.390196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:54.400022 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1384/2773] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:54.945227 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:54.946932 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:55.020101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedacf31-d33c-4893-98e2-2aa802c94681 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:55.025101 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:55.025774 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:55.026060 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:55.026339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:55.028106 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:55.039571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:55.041184 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:55.051443 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1390/2774] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:37:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:37:55.950604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:56.016575 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891614b-4b52-4db6-b70a-33ca3acf72ca None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:56.021269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:56.022142 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:56.022426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:56.022706 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] PATH_INFO: `/v3/users/f6ef4f5010364f1481a71cb1ab18d4d5` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:56.023890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ef4f5010364f1481a71cb1ab18d4d5)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:56.025889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:56.255834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ef4f5010364f1481a71cb1ab18d4d5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.256450 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ef4f5010364f1481a71cb1ab18d4d5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.268591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ef4f5010364f1481a71cb1ab18d4d5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.276188 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ef4f5010364f1481a71cb1ab18d4d5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.355471 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6f6d2e47-38c8-430c-b34f-80ac2fc361e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ef4f5010364f1481a71cb1ab18d4d5'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.362133 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1385/2775] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:37:55 2022] DELETE /identity/v3/users/f6ef4f5010364f1481a71cb1ab18d4d5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:56.639990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:56.651917 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1256d-d87b-4100-b66d-e11d5bcd1cbd service placement] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:56.661457 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:56.712546 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:56.713447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:56.713759 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:56.714065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] PATH_INFO: `/v3/projects/a7147d7d71f34eb9bed7849d7b670d48` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:56.715278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7147d7d71f34eb9bed7849d7b670d48)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:56.717272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:56.733850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:56.734149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:56.734441 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:56.752104 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] MFA Rules not processed for user `7a647aab17cf4345800cd9072a7820c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:56.753379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:56.809542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1386/2776] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:37:56.849691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7147d7d71f34eb9bed7849d7b670d48', 'request_id': 'req-d8131f93-7929-4d3c-9c24-8f5b06da24ab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.857456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7147d7d71f34eb9bed7849d7b670d48', 'request_id': 'req-d8131f93-7929-4d3c-9c24-8f5b06da24ab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.975923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7147d7d71f34eb9bed7849d7b670d48', 'request_id': 'req-d8131f93-7929-4d3c-9c24-8f5b06da24ab'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.987708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] The token cache is being invalidate because project a7147d7d71f34eb9bed7849d7b670d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:37:56.987998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.988731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:56.990893 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1391/2777] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:37:56 2022] DELETE /identity/v3/projects/a7147d7d71f34eb9bed7849d7b670d48 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:56.994879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:56.997327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:57.011982 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba24048-807b-49cb-b23c-3dad3836d23e None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:57.029161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:57.070444 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:57.072701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:57.072701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:57.072701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] PATH_INFO: `/v3/users/6fa5255eb8e3471b82492b46ce940ea3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:57.073264 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa5255eb8e3471b82492b46ce940ea3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:57.075243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:57.340906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa5255eb8e3471b82492b46ce940ea3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:57.341559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa5255eb8e3471b82492b46ce940ea3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:57.354495 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa5255eb8e3471b82492b46ce940ea3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:57.361346 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa5255eb8e3471b82492b46ce940ea3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:57.543961 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-4a4add0c-4325-42bc-abce-578837749491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa5255eb8e3471b82492b46ce940ea3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:57.550449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1387/2778] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:37:56 2022] DELETE /identity/v3/users/6fa5255eb8e3471b82492b46ce940ea3 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:57.849019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:57.851305 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:57.866473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8131f93-7929-4d3c-9c24-8f5b06da24ab admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:57.925407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:57.926284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:57.927090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:57.927372 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] PATH_INFO: `/v3/projects/70c4035711fd4c1aba593dea5174ee92` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:57.928670 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c4035711fd4c1aba593dea5174ee92)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:57.930648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.096607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:58.100179 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c4035711fd4c1aba593dea5174ee92', 'request_id': 'req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:58.106682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c4035711fd4c1aba593dea5174ee92', 'request_id': 'req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:58.204554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c4035711fd4c1aba593dea5174ee92', 'request_id': 'req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:58.216119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] The token cache is being invalidate because project 70c4035711fd4c1aba593dea5174ee92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:37:58.216438 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:58.217100 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-8c4e687b-03b5-4b74-a3f4-8c6a3481715f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:37:58.218639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1392/2779] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:37:57 2022] DELETE /identity/v3/projects/70c4035711fd4c1aba593dea5174ee92 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:58.223951 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.224469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.224622 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.244662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:58.246914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:58.307082 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1388/2780] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:58.315322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.315322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.315805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.338165 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:58.343890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:58.347789 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1393/2781] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:58.352236 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.362047 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8f561-ea7a-42fa-a1af-f74d2f228256 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.366792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.367463 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.367756 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.368040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.368849 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:37:58.369162 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.370792 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.383561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1389/2782] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:37:58.387957 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.398187 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6fdc26-cbba-42dc-ad9b-743ce2a5088d None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.402923 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.403574 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.403859 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.404139 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.404899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.406831 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.503868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1394/2783] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:37:58 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:58.508200 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.518735 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ad5b0-43e7-400d-bd82-3e39ddb8619b admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.544600 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:58.577604 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.578292 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.578569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.578876 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.579581 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.581555 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.704874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1390/2784] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:37:58 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:58.709378 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.719755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fa6a-f8e7-41ea-827f-a35decb355bb admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.724469 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.725174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.725461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.725740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.726486 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.728119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.738647 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1395/2785] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:58.743050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.753477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71048a48-fd64-42b7-9ac1-87dd43e85363 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.759113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.759775 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.760062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.760399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.761134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.762847 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2b053-bfe7-4495-b4fb-f000665e63e4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.772986 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1391/2786] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:58.777001 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:58.787834 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fcc5c-d397-48d0-876d-3d25be741c58 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:58.792565 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:58.793454 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.793745 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.794028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] PATH_INFO: `/v3/projects/a622653241c64566a56a958172817b8c/users/6950f25947d84d81b82811223ccb726a/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.809740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a622653241c64566a56a958172817b8c, user_id=6950f25947d84d81b82811223ccb726a, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:58.815207 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:58.904702 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1396/2787] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:58 2022] PUT /identity/v3/projects/a622653241c64566a56a958172817b8c/users/6950f25947d84d81b82811223ccb726a/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:58.915908 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:58.916367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:58.916753 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:58.944655 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] MFA Rules not processed for user `6950f25947d84d81b82811223ccb726a`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:58.945953 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:59.013574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1392/2788] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:59.018058 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.081769 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72806b7-c371-4cc6-b463-91c373394159 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.087035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.087711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.087999 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.088281 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.089029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.090952 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.241112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1397/2789] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:37:59 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:59.245079 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.256481 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553c0dd-f138-462f-945a-48960c853ec2 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.314829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.315591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.315912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.316216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.316965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.319289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.502387 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1393/2790] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:37:59 2022] POST /identity/v3/users => generated 300 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:37:59.506495 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.516844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965527bf-11ca-49f5-968d-4213e5833385 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.521897 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.522557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.522843 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.523185 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.523954 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.525611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.536926 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1398/2791] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:37:59.543443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.553673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc870-d1ad-4348-8987-b9d415672f9d admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.558345 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.559222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.559510 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.559788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] PATH_INFO: `/v3/projects/1d34c22a1aa549cc8dd723b883862f7b/users/56556a27d4a74447989a8c2109325ee3/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.575119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d34c22a1aa549cc8dd723b883862f7b, user_id=56556a27d4a74447989a8c2109325ee3, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.580619 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.705627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:59.711083 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1394/2792] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:59 2022] PUT /identity/v3/projects/1d34c22a1aa549cc8dd723b883862f7b/users/56556a27d4a74447989a8c2109325ee3/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:59.715177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.725286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654351c-4462-4fc3-b29e-f938ff19332f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.766891 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:37:59.782853 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.783585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.783889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.784163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.784948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.786849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007525d2-1e7e-419b-8425-f75740f6603f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.796625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1399/2793] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:37:59.800998 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.811402 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170b3d3-0f40-4d6f-8c28-d97ebe2965c6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:37:59.816840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:37:59.818499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.818776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.819053 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] PATH_INFO: `/v3/projects/1d34c22a1aa549cc8dd723b883862f7b/users/56556a27d4a74447989a8c2109325ee3/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.821167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d34c22a1aa549cc8dd723b883862f7b, user_id=56556a27d4a74447989a8c2109325ee3, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:37:59.823960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:37:59.836368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1395/2794] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:37:59 2022] PUT /identity/v3/projects/1d34c22a1aa549cc8dd723b883862f7b/users/56556a27d4a74447989a8c2109325ee3/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:37:59.847987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:37:59.848350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:37:59.848671 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:37:59.879045 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] MFA Rules not processed for user `56556a27d4a74447989a8c2109325ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:37:59.880801 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:37:59.954178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1400/2795] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:37:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:37:59.965485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:37:59.969839 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:00.027002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab94ed-90ad-4052-adf0-2ce599c588bf admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:00.030881 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:00.034186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:00.034186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:00.034186 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:00.042098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:00.043546 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2727d1-bfd8-465c-90b1-c64e2e438098 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:00.053874 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1396/2796] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:37:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:00.222548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:00.308970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ad949-43ed-4b24-999f-0dbf97c92e0c None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:00.314080 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:00.314679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:00.315147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:00.315391 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] PATH_INFO: `/v3/users/6950f25947d84d81b82811223ccb726a` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:00.316825 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6950f25947d84d81b82811223ccb726a)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:00.319241 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:00.585009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6950f25947d84d81b82811223ccb726a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:00.591905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6950f25947d84d81b82811223ccb726a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:00.600755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:00.606408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6950f25947d84d81b82811223ccb726a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:00.607191 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6950f25947d84d81b82811223ccb726a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:00.607835 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6950f25947d84d81b82811223ccb726a'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:00.688569 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1401/2797] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:00 2022] DELETE /identity/v3/users/6950f25947d84d81b82811223ccb726a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:00.693489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:00.694027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:00.694150 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:00.712220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:00.716888 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:00.776912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1397/2798] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:00.787404 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:00.788858 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:00.847510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a94d15-f564-48e5-a918-6b724a18f772 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:00.851723 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:00.852403 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:00.852693 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:00.852973 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:00.862119 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:00.863557 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:00.874144 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1402/2799] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:01.556024 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:01.557783 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:01.570864 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9dffb-91d9-4cc8-83fa-8fedc0421592 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:01.575606 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:01.582505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:01.582505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:01.582505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] PATH_INFO: `/v3/projects/a622653241c64566a56a958172817b8c` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:01.582505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a622653241c64566a56a958172817b8c)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:01.582505 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:01.753840 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a622653241c64566a56a958172817b8c', 'request_id': 'req-00c7db97-d058-4464-8279-1776ccabf847'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:01.760808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a622653241c64566a56a958172817b8c', 'request_id': 'req-00c7db97-d058-4464-8279-1776ccabf847'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:01.829866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a622653241c64566a56a958172817b8c', 'request_id': 'req-00c7db97-d058-4464-8279-1776ccabf847'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:01.844342 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] The token cache is being invalidate because project a622653241c64566a56a958172817b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:01.844642 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:01.845680 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:01.849917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1398/2800] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:01 2022] DELETE /identity/v3/projects/a622653241c64566a56a958172817b8c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:01.853857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:01.866271 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3cccc-987d-49c7-98d3-bd9fb205c990 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:01.947942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:01.949035 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:01.949269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:01.949548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] PATH_INFO: `/v3/users/56556a27d4a74447989a8c2109325ee3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:01.950795 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56556a27d4a74447989a8c2109325ee3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:01.952816 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:02.268682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56556a27d4a74447989a8c2109325ee3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:02.281261 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56556a27d4a74447989a8c2109325ee3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:02.296402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56556a27d4a74447989a8c2109325ee3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:02.297491 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56556a27d4a74447989a8c2109325ee3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:02.298114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-f18e397f-a9e7-41bc-9256-5f68bbca9379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56556a27d4a74447989a8c2109325ee3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:02.418754 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1403/2801] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:01 2022] DELETE /identity/v3/users/56556a27d4a74447989a8c2109325ee3 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:02.923407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:02.928761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:02.946270 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7db97-d058-4464-8279-1776ccabf847 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:02.954311 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:03.006205 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.007204 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.007500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.008095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] PATH_INFO: `/v3/projects/1d34c22a1aa549cc8dd723b883862f7b` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.009117 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d34c22a1aa549cc8dd723b883862f7b)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.011054 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.140715 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d34c22a1aa549cc8dd723b883862f7b', 'request_id': 'req-f02783a5-0c30-42c8-9b02-af777f0d0171'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:03.150689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d34c22a1aa549cc8dd723b883862f7b', 'request_id': 'req-f02783a5-0c30-42c8-9b02-af777f0d0171'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:03.244334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d34c22a1aa549cc8dd723b883862f7b', 'request_id': 'req-f02783a5-0c30-42c8-9b02-af777f0d0171'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:03.256665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] The token cache is being invalidate because project 1d34c22a1aa549cc8dd723b883862f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:03.256952 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:03.257617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f02783a5-0c30-42c8-9b02-af777f0d0171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:03.259151 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1399/2802] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:02 2022] DELETE /identity/v3/projects/1d34c22a1aa549cc8dd723b883862f7b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:03.264123 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.264665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.264852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.277680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:03.284278 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:03.285577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:03.347383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1404/2803] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:03.356343 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.356635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.356919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.376268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:03.377552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:03.385841 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1400/2804] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:03.390108 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.400235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faadec9-db14-4f8f-a68e-4956655e69cd None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.405202 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.405882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.406172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.406461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.407231 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:38:03.407554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.409226 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.422026 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1405/2805] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:38:03.431550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.440992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee47b28-fa05-46d2-aedf-783f5362bbf3 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.447820 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.448559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.448806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.449084 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.449808 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.451714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.548257 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1401/2806] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:38:03 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:03.553284 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.563429 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab405-ffc1-421d-b1f9-33545ef28544 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.621304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.622272 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.622561 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.622845 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.623541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.625554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.720562 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1406/2807] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:38:03 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:03.725300 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.735866 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951cae4-9dd0-454f-801b-ce5d6e7893f9 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.740868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.741534 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.741827 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.742175 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.742914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.744591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.756012 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1402/2808] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:03.759983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.771071 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee919e87-cc26-4491-870b-b7b4fefccb53 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.781350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.781350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.781350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.781350 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.781794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.781794 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee00819-83f1-4c71-b53c-87025c09e09f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.790740 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1407/2809] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:03.795002 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:03.805522 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64fc0d-ee55-49e7-aee6-0d765305e744 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:03.810362 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:03.811302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.811588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.811896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] PATH_INFO: `/v3/projects/8f9ca9f4cd5b4df586c57c5adb5e28d9/users/8d440c0e71844d58b29770282b18c374/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.827347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9ca9f4cd5b4df586c57c5adb5e28d9, user_id=8d440c0e71844d58b29770282b18c374, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:03.833192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:03.916192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1403/2810] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:03 2022] PUT /identity/v3/projects/8f9ca9f4cd5b4df586c57c5adb5e28d9/users/8d440c0e71844d58b29770282b18c374/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:03.927742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:03.928034 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:03.928327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:03.955878 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] MFA Rules not processed for user `8d440c0e71844d58b29770282b18c374`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:03.957212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:04.022383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1408/2811] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:04.027122 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.075058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:04.095178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80834efb-3f6d-4dcd-b5e1-773606394f86 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.099958 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.100658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.100942 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.101220 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.101914 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.103872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.219661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1404/2812] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:38:04 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:04.224126 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.231510 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:04.235752 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1020002-1f0a-42a0-9c6f-264f81080b23 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.295861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.298038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.298038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.298038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.298038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.299736 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.407217 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1409/2813] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:38:04 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:04.411542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.421993 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a45df-9b9d-4cc9-bb2b-c92f91bac783 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.427143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.428044 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.428358 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.428643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.429376 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.430996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.441545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1405/2814] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:38:04.446571 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.456856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07060b-184a-40fc-82ee-d5514970a658 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.461506 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.462383 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.462667 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.462945 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] PATH_INFO: `/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014/users/dc8378514329496897e371ca9d38f9d0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.478889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47c7b156b1b423ebf8e4c5022d9b014, user_id=dc8378514329496897e371ca9d38f9d0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.481712 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.601132 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1410/2815] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:04 2022] PUT /identity/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014/users/dc8378514329496897e371ca9d38f9d0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:04.605256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.616353 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667f70e-4752-4ae9-8939-61147969286c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.675747 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.676480 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.676807 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.677144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.677882 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.679618 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e506d-a3b8-4911-8517-14117ec63b3e admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.689608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1406/2816] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:38:04.693881 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.704384 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98da5ae-79da-447d-8f22-58e43a50847f admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.709737 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.710648 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.710935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.711247 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] PATH_INFO: `/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014/users/dc8378514329496897e371ca9d38f9d0/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.713304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47c7b156b1b423ebf8e4c5022d9b014, user_id=dc8378514329496897e371ca9d38f9d0, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.716007 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.727052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1411/2817] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:04 2022] PUT /identity/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014/users/dc8378514329496897e371ca9d38f9d0/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:04.788727 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.789018 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.789301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.815088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] MFA Rules not processed for user `dc8378514329496897e371ca9d38f9d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:04.816334 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:04.884098 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1407/2818] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:04.894668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:04.897044 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:04.953857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623614d-79c0-4e7c-a330-b7f13cb4fe32 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:04.957424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:04.958836 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:04.959186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:04.959470 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:04.968516 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:04.969942 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d176217-6601-4a6b-9a6d-483898feff01 service cinder] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:04.979974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1412/2819] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:38:06.252301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:06.292645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:06.315268 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf385906-5b02-4308-9ee3-6ebdcd6da860 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:06.319856 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:06.320762 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:06.321051 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:06.321330 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] PATH_INFO: `/v3/users/8d440c0e71844d58b29770282b18c374` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:06.322515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d440c0e71844d58b29770282b18c374)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:06.324477 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:06.543838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d440c0e71844d58b29770282b18c374'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:06.544538 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d440c0e71844d58b29770282b18c374'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:06.557256 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d440c0e71844d58b29770282b18c374'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:06.564101 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d440c0e71844d58b29770282b18c374'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:06.655147 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d440c0e71844d58b29770282b18c374'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:06.662274 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1408/2820] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:06 2022] DELETE /identity/v3/users/8d440c0e71844d58b29770282b18c374 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:06.666862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:06.667376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:06.667480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:06.685498 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:06.686715 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:06.736861 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:06.746219 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1413/2821] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:06.756323 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:06.757906 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:06.815484 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235229af-6a47-418d-a21b-6a18a35512f7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:06.819182 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:06.819829 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:06.820583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:06.820583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:06.829574 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:06.831156 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:06.841320 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1409/2822] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:07.305499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:07.307262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:07.320268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509637d-e020-4a5b-acdf-3c4e9c25e7b2 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:07.325088 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:07.325961 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:07.326432 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:07.326720 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] PATH_INFO: `/v3/projects/8f9ca9f4cd5b4df586c57c5adb5e28d9` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:07.327948 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9ca9f4cd5b4df586c57c5adb5e28d9)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:07.329934 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:07.445857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9ca9f4cd5b4df586c57c5adb5e28d9', 'request_id': 'req-4ce954db-7ada-4bea-af40-b4f141a63c0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.452479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9ca9f4cd5b4df586c57c5adb5e28d9', 'request_id': 'req-4ce954db-7ada-4bea-af40-b4f141a63c0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.523915 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9ca9f4cd5b4df586c57c5adb5e28d9', 'request_id': 'req-4ce954db-7ada-4bea-af40-b4f141a63c0b'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.535072 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] The token cache is being invalidate because project 8f9ca9f4cd5b4df586c57c5adb5e28d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:07.535357 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.535998 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.537706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1414/2823] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:07 2022] DELETE /identity/v3/projects/8f9ca9f4cd5b4df586c57c5adb5e28d9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:38:07.541776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:07.551885 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde9595-a1fd-4a56-bf30-a418353ecb64 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:07.604457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:07.610224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:07.611095 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:07.611380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:07.611661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] PATH_INFO: `/v3/users/dc8378514329496897e371ca9d38f9d0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:07.612994 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8378514329496897e371ca9d38f9d0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:07.615632 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:07.848872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8378514329496897e371ca9d38f9d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.849922 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8378514329496897e371ca9d38f9d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.861990 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8378514329496897e371ca9d38f9d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:07.869623 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8378514329496897e371ca9d38f9d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.006893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-863ee8bb-64ea-46f8-84d0-3b07db501208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8378514329496897e371ca9d38f9d0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.013498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1410/2824] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:07 2022] DELETE /identity/v3/users/dc8378514329496897e371ca9d38f9d0 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:08.540149 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:08.541757 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:08.556479 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce954db-7ada-4bea-af40-b4f141a63c0b admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:08.615451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:08.616507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:08.616797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:08.617074 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] PATH_INFO: `/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:08.618319 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47c7b156b1b423ebf8e4c5022d9b014)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:08.624269 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:08.763665 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47c7b156b1b423ebf8e4c5022d9b014', 'request_id': 'req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.769981 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47c7b156b1b423ebf8e4c5022d9b014', 'request_id': 'req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.838048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47c7b156b1b423ebf8e4c5022d9b014', 'request_id': 'req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.851253 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] The token cache is being invalidate because project a47c7b156b1b423ebf8e4c5022d9b014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:08.851856 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.852636 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-2371ff4a-6bdf-489d-8862-7c86a43b3d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:08.854643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1415/2825] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:08 2022] DELETE /identity/v3/projects/a47c7b156b1b423ebf8e4c5022d9b014 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:08.860442 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:08.860981 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:08.861129 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:08.881643 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:08.883006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:08.948203 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1411/2826] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:08.955982 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:08.956282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:08.956604 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:08.978308 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:08.979639 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:08.988094 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1416/2827] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:08.998627 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.008985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb5b6d-2d59-4959-a14f-fada9ccf7d06 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.013511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.023273 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.029878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1412/2828] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:38:09.034544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.041189 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:09.045157 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeae8b8-e74e-41a1-8f1c-907865b37a88 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.049983 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.050649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.050935 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.051216 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.051925 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.054174 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.139852 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1417/2829] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:38:09 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:09.144183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.155041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4de36d-624a-46e3-8b03-5b35f57b1af6 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.212036 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.212798 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.213049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.213332 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.214027 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.215967 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.225562 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:09.301802 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1413/2830] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:38:09 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 18:38:09.306462 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.316990 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adde70-9396-4c18-854b-c23e9ed0c661 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.322814 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.323541 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.323838 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.324125 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.324907 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.326529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.337626 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1418/2831] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:09.341957 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.352890 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7c9cf-27a0-4d7f-8b34-e026d094bb7c admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.357714 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.358390 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.358689 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.359007 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.359788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.361499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6eba1-98eb-4a74-b92a-5723b327fbd4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.373693 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1414/2832] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:09.377956 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.388587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603a982-9225-4173-82a9-cfe885809acc admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.393167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.394115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.394402 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.394682 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] PATH_INFO: `/v3/projects/699e9fad08c64a0984ba7c04cb6b165f/users/ebdf0830816d474d89b943dbf1d032e3/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.410235 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699e9fad08c64a0984ba7c04cb6b165f, user_id=ebdf0830816d474d89b943dbf1d032e3, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.413047 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.484148 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1419/2833] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:09 2022] PUT /identity/v3/projects/699e9fad08c64a0984ba7c04cb6b165f/users/ebdf0830816d474d89b943dbf1d032e3/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:09.495685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.496010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.496299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.523803 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] MFA Rules not processed for user `ebdf0830816d474d89b943dbf1d032e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:09.525062 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:09.590233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1415/2834] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:09.594588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.657257 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979087f0-1ec5-449e-a174-7c48530c4259 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.662602 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.663264 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.663544 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.663818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.664550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.666452 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.742306 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1420/2835] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:38:09 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:09.746392 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.757470 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13243c-4276-42b3-a912-15638510b5c4 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.813586 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.814261 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.814552 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.814836 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.815577 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.817557 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.919761 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1416/2836] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:38:09 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:09.923776 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.934848 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43c84c-aa07-4db7-ab31-6719550b028e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.939756 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.940455 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.940748 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.941027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.941765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.944085 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:09.954811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1421/2837] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 18:38:09.959917 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:09.969825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e056-8bbb-4a04-894d-9778e3ce2f63 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:09.974974 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:09.975886 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:09.976169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:09.976493 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] PATH_INFO: `/v3/projects/b098774490d4446e8293702fe82db660/users/7d6e02dc05ba42f1bf227e59957fd242/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:09.991825 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b098774490d4446e8293702fe82db660, user_id=7d6e02dc05ba42f1bf227e59957fd242, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:09.994529 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:10.110260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1417/2838] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:09 2022] PUT /identity/v3/projects/b098774490d4446e8293702fe82db660/users/7d6e02dc05ba42f1bf227e59957fd242/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:10.118286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:10.125941 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e1719-a2c6-45dc-8ac7-f1124ebb2501 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:10.184668 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:10.185262 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:10.185548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:10.185827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:10.186556 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:10.188159 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67799942-5be9-43e9-a804-34a80fe2d9cc admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:10.198993 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1422/2839] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:10.203280 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:10.214435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78adaf-cd69-4202-86d0-d13bdb3935d7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:10.219160 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:10.220035 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:10.220347 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:10.220635 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] PATH_INFO: `/v3/projects/b098774490d4446e8293702fe82db660/users/7d6e02dc05ba42f1bf227e59957fd242/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:10.222701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b098774490d4446e8293702fe82db660, user_id=7d6e02dc05ba42f1bf227e59957fd242, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:10.225409 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:10.235721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1418/2840] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:10 2022] PUT /identity/v3/projects/b098774490d4446e8293702fe82db660/users/7d6e02dc05ba42f1bf227e59957fd242/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:10.246246 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:10.246862 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:10.247137 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:10.275019 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] MFA Rules not processed for user `7d6e02dc05ba42f1bf227e59957fd242`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:10.276228 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:10.281193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:10.344342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1423/2841] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:10.355096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:10.356834 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:10.385460 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:10.415472 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422aa351-e3ad-4df3-9756-ea385ea74443 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:10.419868 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:10.420554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:10.420848 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:10.421131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:10.431449 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:10.432992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d3f3d-1a75-4783-9370-81255f7c663d service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:10.442614 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1419/2842] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:10.953330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:11.019086 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769833a-68f8-4383-b5f0-39e8088f4b19 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:11.023908 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:11.024823 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:11.025115 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:11.025398 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] PATH_INFO: `/v3/users/ebdf0830816d474d89b943dbf1d032e3` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:11.026614 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdf0830816d474d89b943dbf1d032e3)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:11.028642 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:11.241729 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdf0830816d474d89b943dbf1d032e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:11.250146 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdf0830816d474d89b943dbf1d032e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:11.272473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdf0830816d474d89b943dbf1d032e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:11.273366 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdf0830816d474d89b943dbf1d032e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:11.274078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdf0830816d474d89b943dbf1d032e3'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:11.350408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1424/2843] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:10 2022] DELETE /identity/v3/users/ebdf0830816d474d89b943dbf1d032e3 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:11.355169 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:11.355658 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:11.355757 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:11.375725 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:11.377058 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:11.438317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1420/2844] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:11.448992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:11.450186 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:11.479631 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:11.513705 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c4d2d-0aba-4583-a3f2-0dc0024c76af admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:11.517608 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:11.518268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:11.518627 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:11.518819 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:11.529779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:11.531453 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:11.542212 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1425/2845] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:12.002743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:12.004239 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:12.017607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeda693-60a6-428d-b510-bfbe714c826d None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:12.022489 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:12.023469 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:12.023776 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:12.024041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] PATH_INFO: `/v3/projects/699e9fad08c64a0984ba7c04cb6b165f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:12.025393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699e9fad08c64a0984ba7c04cb6b165f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:12.027404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:12.164900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699e9fad08c64a0984ba7c04cb6b165f', 'request_id': 'req-f52927fd-eef3-4608-bbde-f8e1bac5ca43'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.171107 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699e9fad08c64a0984ba7c04cb6b165f', 'request_id': 'req-f52927fd-eef3-4608-bbde-f8e1bac5ca43'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.257965 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699e9fad08c64a0984ba7c04cb6b165f', 'request_id': 'req-f52927fd-eef3-4608-bbde-f8e1bac5ca43'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.269288 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] The token cache is being invalidate because project 699e9fad08c64a0984ba7c04cb6b165f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:12.269578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.270237 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.271872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1421/2846] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:12 2022] DELETE /identity/v3/projects/699e9fad08c64a0984ba7c04cb6b165f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:12.275967 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:12.286025 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625a4f-94f2-432a-af02-a030ce722269 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:12.340559 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:12.342813 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:12.347867 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:12.348782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:12.349588 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:12.349872 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] PATH_INFO: `/v3/users/7d6e02dc05ba42f1bf227e59957fd242` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:12.351797 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6e02dc05ba42f1bf227e59957fd242)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:12.353421 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:12.405123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:12.429438 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52927fd-eef3-4608-bbde-f8e1bac5ca43 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:12.435412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:12.436076 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:12.436398 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:12.436685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:12.438525 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:12.499111 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:12.500816 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:12.510668 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1422/2847] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:38:12.587490 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6e02dc05ba42f1bf227e59957fd242'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.599654 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6e02dc05ba42f1bf227e59957fd242'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.611548 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6e02dc05ba42f1bf227e59957fd242'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.612282 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6e02dc05ba42f1bf227e59957fd242'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.612987 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-759e6589-f3d4-4c8b-ad5e-49a252cfc00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6e02dc05ba42f1bf227e59957fd242'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:12.703706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1426/2848] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:38:12 2022] DELETE /identity/v3/users/7d6e02dc05ba42f1bf227e59957fd242 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:13.233884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:13.233884 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:13.250789 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78468dc-e97b-48b0-9df8-a53aca90bb23 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:13.316196 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:13.317233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.317560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.318011 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] PATH_INFO: `/v3/projects/b098774490d4446e8293702fe82db660` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.319457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b098774490d4446e8293702fe82db660)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:13.321487 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:13.453049 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b098774490d4446e8293702fe82db660', 'request_id': 'req-6acec914-ead1-4dd6-b49d-90f779d80f69'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:13.459978 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b098774490d4446e8293702fe82db660', 'request_id': 'req-6acec914-ead1-4dd6-b49d-90f779d80f69'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:13.554878 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b098774490d4446e8293702fe82db660', 'request_id': 'req-6acec914-ead1-4dd6-b49d-90f779d80f69'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:13.567299 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] The token cache is being invalidate because project b098774490d4446e8293702fe82db660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:38:13.567590 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:13.568412 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-6acec914-ead1-4dd6-b49d-90f779d80f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:38:13.569960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1423/2849] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:38:13 2022] DELETE /identity/v3/projects/b098774490d4446e8293702fe82db660 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:13.574992 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.575461 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.575735 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.595328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:13.596833 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:13.639589 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:13.667083 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1427/2850] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:13.670919 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.671216 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.671498 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.689854 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:13.691041 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:13.699178 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1424/2851] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:13.707590 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:13.714379 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c3f26-0d71-499e-be93-93533bfde625 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:13.719327 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:13.720029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.720345 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.720630 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] PATH_INFO: `/v3/domains` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.721374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80576) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:38:13.721706 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:13.723322 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:13.735617 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1428/2852] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:38:13.740561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:13.750645 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa207325-3c8b-469f-9f2e-aad013ccbd93 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:13.756118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:13.756875 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.757161 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.757440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.759701 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:13.760131 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:13.769020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:13.869897 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1425/2853] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:38:13 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:13.878441 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:13.888493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf9dec-58a2-4598-8d97-20c6db19e641 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:13.949066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:13.949764 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:13.950036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:13.950321 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:13.951052 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:13.953038 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:14.050082 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1429/2854] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:38:13 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:38:14.054791 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:14.065474 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d4e9f-f9e0-4fbe-ae88-2773281df3e2 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:14.070500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:14.071279 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:14.071570 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:14.071851 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:14.072654 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:14.074266 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:14.084360 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1426/2855] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:14.090439 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:14.101112 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc6641-b51b-4e32-a8cc-369b4c81e3f4 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:14.105989 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:14.106679 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:14.106970 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:14.107255 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:14.108054 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:14.109730 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e310f-78df-44f9-a50e-e99ec89b89e7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:14.121193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1430/2856] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:38:14.125417 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:14.135898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc9e6a-ad03-476b-8d43-64b0bd6913b4 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:14.140858 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:14.141743 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:14.142023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:14.142285 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] PATH_INFO: `/v3/projects/072667f3650243fc9973a36efad7ae1d/users/84f9e5a76bbe4dcf906a1b46f53fa7c3/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:14.157900 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] RBAC: Authorizing `identity:create_grant(project_id=072667f3650243fc9973a36efad7ae1d, user_id=84f9e5a76bbe4dcf906a1b46f53fa7c3, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:14.160621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:14.267222 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1427/2857] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:38:14 2022] PUT /identity/v3/projects/072667f3650243fc9973a36efad7ae1d/users/84f9e5a76bbe4dcf906a1b46f53fa7c3/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:38:14.276714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7910f-52ed-4a2b-858d-5bcaf1575da1 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:14.277009 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7910f-52ed-4a2b-858d-5bcaf1575da1 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:14.277292 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7910f-52ed-4a2b-858d-5bcaf1575da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:14.297762 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-06f7910f-52ed-4a2b-858d-5bcaf1575da1 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:14.298978 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-06f7910f-52ed-4a2b-858d-5bcaf1575da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:14.358385 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1431/2858] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:14.368163 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:14.369916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:14.427879 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330900ad-d53e-44c9-a299-8cabd8ceb7df admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:14.431676 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:14.432349 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:14.432646 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:14.432927 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:14.442516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:14.444289 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:14.454445 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1428/2859] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:15.923172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c44e-df92-41f7-a4a7-b546cfde502d None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:15.923785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c44e-df92-41f7-a4a7-b546cfde502d None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:15.923785 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c44e-df92-41f7-a4a7-b546cfde502d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:15.967508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-ae68c44e-df92-41f7-a4a7-b546cfde502d None None] MFA Rules not processed for user `84f9e5a76bbe4dcf906a1b46f53fa7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:15.967508 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-ae68c44e-df92-41f7-a4a7-b546cfde502d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:16.009704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:16.045193 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1432/2860] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:16.058905 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:16.060621 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:16.124464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e113449-82b7-4709-8475-93d451379c70 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:16.128121 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fe9acbbbdc463ab22869886df4e3aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:16.128812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:16.129113 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:16.129394 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:16.138269 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:16.139793 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:16.149210 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1429/2861] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:38:17.951114 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36ee1-c6cf-4f72-a770-4e093f4cf88e None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:17.951612 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36ee1-c6cf-4f72-a770-4e093f4cf88e None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:17.951821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c36ee1-c6cf-4f72-a770-4e093f4cf88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:17.970827 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-13c36ee1-c6cf-4f72-a770-4e093f4cf88e None None] MFA Rules not processed for user `84f9e5a76bbe4dcf906a1b46f53fa7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:17.975685 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-13c36ee1-c6cf-4f72-a770-4e093f4cf88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:17.981066 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1433/2862] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:26.144771 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:27.965158 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=12 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:35.677407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:38:35.679144 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:35.757302 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d37ebc-ee72-4c02-852c-6055fe191224 service cinder] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:38:35.762335 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:38:35.762992 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:35.763286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:35.763566 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:35.765411 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:35.778361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:38:35.780023 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6893c325-ed75-432e-a9a4-dd37736f9186 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:38:35.790759 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1430/2863] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:38:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:38:37.627177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:38:37.627690 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:38:37.627811 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:38:37.645145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] MFA Rules not processed for user `84f9e5a76bbe4dcf906a1b46f53fa7c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:38:37.646346 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:38:37.653704 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1434/2864] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:38:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:38:45.786055 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=32 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:38:47.640079 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:00.990485 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9240e89-bf79-4865-975f-17e164544c7c None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:00.991026 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9240e89-bf79-4865-975f-17e164544c7c None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:00.991187 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9240e89-bf79-4865-975f-17e164544c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:01.060167 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a9240e89-bf79-4865-975f-17e164544c7c None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:01.061446 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a9240e89-bf79-4865-975f-17e164544c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:01.131020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1431/2865] 10.222.0.47 () {58 vars in 1009 bytes} [Tue Jul 12 18:39:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:39:01.302623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:01.304456 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:01.408719 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d3120-6983-4b3a-800f-25a95021aaeb None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:01.417701 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:01.418808 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:01.419302 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:01.419792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:01.434930 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:01.437661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:01.452905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1435/2866] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:39:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:39:11.128046 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=44 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:11.445779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=41 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:18.806814 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:18.807327 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:18.807439 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:18.826293 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] MFA Rules not processed for user `7e4dbbeee2c847118b3fc42ae7c892f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:18.827506 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:18.834733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1432/2867] 10.222.0.47 () {58 vars in 967 bytes} [Tue Jul 12 18:39:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:18.848091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:18.858937 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19f8ec-7c27-4524-b157-831089504799 service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:18.863938 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe3f01883b804652b396cd4e28ab4b46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:18.864611 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:18.864899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:18.865182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:18.874641 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:18.876230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:18.885087 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1436/2868] 10.222.0.47 () {60 vars in 1384 bytes} [Tue Jul 12 18:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 18:39:21.764911 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:21.777221 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ce259-c9e9-4c9d-91bc-00c39e91a4bd None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:21.781722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:21.782591 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:21.782872 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:21.783149 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] PATH_INFO: `/v3/users/84f9e5a76bbe4dcf906a1b46f53fa7c3` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:21.784404 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f9e5a76bbe4dcf906a1b46f53fa7c3)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:21.786465 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:21.985665 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f9e5a76bbe4dcf906a1b46f53fa7c3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:21.986427 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f9e5a76bbe4dcf906a1b46f53fa7c3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.000050 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f9e5a76bbe4dcf906a1b46f53fa7c3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.007298 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f9e5a76bbe4dcf906a1b46f53fa7c3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.100838 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-a87127eb-dddd-4d22-b789-681207dd4c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f9e5a76bbe4dcf906a1b46f53fa7c3'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.108440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1433/2869] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:39:21 2022] DELETE /identity/v3/users/84f9e5a76bbe4dcf906a1b46f53fa7c3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:39:22.395304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:22.412554 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb70a23-eb81-461c-b98b-2624df70094d service swift] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:22.478407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:22.479294 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:22.479585 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:22.479870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] PATH_INFO: `/v3/projects/072667f3650243fc9973a36efad7ae1d` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:22.481178 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=072667f3650243fc9973a36efad7ae1d)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:22.483138 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:22.578765 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072667f3650243fc9973a36efad7ae1d', 'request_id': 'req-a7a90779-9e96-4c3d-8657-d13fa098df52'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.585480 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072667f3650243fc9973a36efad7ae1d', 'request_id': 'req-a7a90779-9e96-4c3d-8657-d13fa098df52'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.671974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072667f3650243fc9973a36efad7ae1d', 'request_id': 'req-a7a90779-9e96-4c3d-8657-d13fa098df52'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.684974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] The token cache is being invalidate because project 072667f3650243fc9973a36efad7ae1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:39:22.685268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.685959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-a7a90779-9e96-4c3d-8657-d13fa098df52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:39:22.687534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1437/2870] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:39:22 2022] DELETE /identity/v3/projects/072667f3650243fc9973a36efad7ae1d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:39:22.692806 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:22.693103 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:22.693482 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:22.713588 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:22.714983 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:22.777985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1434/2871] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:22.786030 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:22.786362 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:22.786651 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:22.806068 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:22.807890 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:22.816147 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1438/2872] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 18:39:22.820637 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:22.831100 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ec87c-0910-470d-b70a-a13d71586955 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:22.835709 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:22.836407 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:22.836702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:22.836985 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:22.838379 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80577) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 18:39:22.838702 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:22.844260 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:22.853542 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1435/2873] 10.222.0.47 () {58 vars in 1152 bytes} [Tue Jul 12 18:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 18:39:22.858084 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:22.867568 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e383d-5e66-415f-9512-96c3c9ae95ad None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:22.873959 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:22.874625 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:22.874910 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:22.875842 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:22.875972 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:22.877916 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:22.996883 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1439/2874] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:39:22 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:39:23.001418 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:23.012589 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58de93b-3bfa-4cce-a747-52b475515ec8 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:23.072569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:23.081916 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:23.082694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.083032 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.083317 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.084015 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:23.086105 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:23.211641 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1436/2875] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:39:23 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:39:23.216286 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:23.227036 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa59c9-1661-4ac4-b620-e8167efddff7 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:23.232573 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:23.233250 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.233538 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.233821 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.234563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:23.236176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:23.247266 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1440/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:39:23.251662 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:23.262401 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd1698-f35c-4bbd-8842-75db4f277065 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:23.267152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:23.267855 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.268143 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.268459 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] PATH_INFO: `/v3/roles` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.269207 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:23.270850 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f89dde-a0a6-4982-8ec1-d9ac0ac6551b admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:23.282518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1437/2877] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:39:23.286734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:23.297960 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef817295-6d60-4fcd-a185-69a227589734 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:23.303256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:23.304240 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] REQUEST_METHOD: `PUT` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.304680 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.305024 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] PATH_INFO: `/v3/projects/cbb64781270b435aaf0eb5e2c4d290b0/users/11c8efca90fa4e309ada93f3eecda4cc/roles/5a2774adec8f4206a81f758406367b9f` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.323687 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb64781270b435aaf0eb5e2c4d290b0, user_id=11c8efca90fa4e309ada93f3eecda4cc, role_id=5a2774adec8f4206a81f758406367b9f)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:23.326529 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:23.400721 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1441/2878] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:39:23 2022] PUT /identity/v3/projects/cbb64781270b435aaf0eb5e2c4d290b0/users/11c8efca90fa4e309ada93f3eecda4cc/roles/5a2774adec8f4206a81f758406367b9f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:39:23.406443 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5400-254c-4b84-8c68-237ffdd9e756 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.406812 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5400-254c-4b84-8c68-237ffdd9e756 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.407253 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d5400-254c-4b84-8c68-237ffdd9e756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.427240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-0d2d5400-254c-4b84-8c68-237ffdd9e756 None None] MFA Rules not processed for user `26ab52b942da412ea520d26ff61916e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:23.428511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0d2d5400-254c-4b84-8c68-237ffdd9e756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:23.490896 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1438/2879] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:23.500984 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:23.502711 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:23.517792 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:23.561256 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d83d2d-65ed-486b-ad59-083a91defaab admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:23.564899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:23.572442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:23.572442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:23.572442 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:23.576029 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:23.577507 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:23.588599 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1442/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:39:29.488070 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:29.488722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:29.488722 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:29.521569 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.auth.core [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] MFA Rules not processed for user `11c8efca90fa4e309ada93f3eecda4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80577) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:29.522964 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:29.597240 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1439/2881] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:29.601905 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:29.614105 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc6752-48fc-4b35-bd29-544807f4029e service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:29.621873 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:29.623090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:29.623525 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:29.623944 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:29.624900 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:29.627167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:29.727081 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1443/2882] 10.222.0.47 () {60 vars in 1152 bytes} [Tue Jul 12 18:39:29 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:39:29.731694 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:29.744563 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fd143-6b93-4a61-a63b-78fa8fac0173 None None] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:29.813382 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:29.814077 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] REQUEST_METHOD: `POST` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:29.814660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:29.814660 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] PATH_INFO: `/v3/users` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:29.815447 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:29.817499 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:29.826368 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:29.936893 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1440/2883] 10.222.0.47 () {60 vars in 1143 bytes} [Tue Jul 12 18:39:29 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 18:39:29.940847 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:29.951857 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8f03e-953e-44f7-b44a-8151fa57a3f9 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:29.956698 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:29.957373 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:29.957663 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:29.957947 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:29.958691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:29.960330 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:29.972110 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1444/2884] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:39:29.976440 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:29.987458 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973ce8-cd53-47ce-afd9-f0008f3b256a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:29.992127 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:29.993069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:29.993359 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:29.993673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] PATH_INFO: `/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484/users/fb822c5499924685864f4d506bebf381/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:30.009995 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdb32f2e54e4cfe8a8d2441f0ac4484, user_id=fb822c5499924685864f4d506bebf381, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:30.012773 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:30.085396 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1441/2885] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:39:29 2022] PUT /identity/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484/users/fb822c5499924685864f4d506bebf381/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:39:30.090493 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:30.100784 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9533e2-41bb-4ed9-8687-e0d360a99c3e admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:30.161742 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:30.162426 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:30.162722 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:30.163006 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:30.163755 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:30.165550 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc34b6-0898-4f7c-948a-14e101a36bf7 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:30.176096 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1445/2886] 10.222.0.47 () {58 vars in 1121 bytes} [Tue Jul 12 18:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 18:39:30.180251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:30.191056 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0572c6f-3bcd-436b-8c86-16619ffe6fe7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:30.195628 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:30.196543 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] REQUEST_METHOD: `PUT` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:30.197420 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:30.197698 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] PATH_INFO: `/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484/users/fb822c5499924685864f4d506bebf381/roles/be70b8e955f74d66a3dd5d81b794548e` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:30.199730 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdb32f2e54e4cfe8a8d2441f0ac4484, user_id=fb822c5499924685864f4d506bebf381, role_id=be70b8e955f74d66a3dd5d81b794548e)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:30.202423 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:30.214126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1442/2887] 10.222.0.47 () {60 vars in 1482 bytes} [Tue Jul 12 18:39:30 2022] PUT /identity/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484/users/fb822c5499924685864f4d506bebf381/roles/be70b8e955f74d66a3dd5d81b794548e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:39:36.228408 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443a7d1-0d68-4872-9aff-b22f9178ca33 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:36.229065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443a7d1-0d68-4872-9aff-b22f9178ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:36.229065 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443a7d1-0d68-4872-9aff-b22f9178ca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:36.258028 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-b443a7d1-0d68-4872-9aff-b22f9178ca33 None None] MFA Rules not processed for user `fb822c5499924685864f4d506bebf381`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:36.259830 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-b443a7d1-0d68-4872-9aff-b22f9178ca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:36.333817 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1446/2888] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:36.349361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:39:36.351155 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:36.428385 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9e1db-b519-4348-8b33-01e85bbd184a admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:39:36.434823 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:39:36.435617 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:36.435805 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:36.436088 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:36.502576 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:39:36.504246 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:39:36.514531 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1443/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:39:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:39:39.209172 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] REQUEST_METHOD: `POST` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:39:39.209649 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:39:39.209758 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:39:39.217783 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:39.230559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.auth.core [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] MFA Rules not processed for user `11c8efca90fa4e309ada93f3eecda4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80576) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 18:39:39.232244 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:39:39.243577 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1447/2890] 10.222.0.47 () {58 vars in 957 bytes} [Tue Jul 12 18:39:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 18:39:46.509456 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=4,keystone:SELECT=73 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:39:49.312015 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:40:10.626430 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:10.630009 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:10.697172 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b878b-a19d-4056-8d8a-bd7b7b08128b service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:10.701168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:10.701843 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:10.702134 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:10.702416 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:10.704348 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:10.721224 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:10.724744 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:10.738889 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1444/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:40:13.837779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:13.849304 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de413c-03fa-48e3-b69e-e9c5141ab318 None None] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:13.853205 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:13.853870 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:13.854155 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:13.854431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:13.856368 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:13.927268 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:13.928822 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:13.940400 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1448/2892] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:40:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:40:14.381361 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:14.391511 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634de37-06a5-447a-9baa-dd87ef3ad27c service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:14.395561 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:14.400672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:14.400672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:14.400672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:14.400672 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:14.415380 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:14.417393 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:14.430059 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1445/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:40:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:40:19.315018 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:19.329919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8fbe3-dd40-4fee-af6c-ac4f581b2238 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:19.333874 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:19.334559 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:19.334849 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:19.335130 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:19.347316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:19.349337 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:19.361245 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1449/2894] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:40:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:40:24.424545 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=32 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:40:24.451367 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:24.453158 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:24.472941 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a40986-608d-4801-ae19-cb25acd65a85 service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:24.478700 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:24.479616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:24.479898 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:24.480275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] PATH_INFO: `/v3/users/7a647aab17cf4345800cd9072a7820c0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:24.481583 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a647aab17cf4345800cd9072a7820c0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:24.483608 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:24.688275 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a647aab17cf4345800cd9072a7820c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:24.689000 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a647aab17cf4345800cd9072a7820c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:24.706374 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a647aab17cf4345800cd9072a7820c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:24.719403 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a647aab17cf4345800cd9072a7820c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:24.792052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a647aab17cf4345800cd9072a7820c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:24.800020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1446/2895] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:40:24 2022] DELETE /identity/v3/users/7a647aab17cf4345800cd9072a7820c0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:40:25.095316 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:25.111374 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a84e7-b5c4-4502-bcc0-9096f1766dca service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:25.212691 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:25.213899 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:25.214176 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:25.214644 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] PATH_INFO: `/v3/projects/447abd08211d49f9b503130ac5c380b7` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:25.216013 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=447abd08211d49f9b503130ac5c380b7)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:25.218027 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:25.330876 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '447abd08211d49f9b503130ac5c380b7', 'request_id': 'req-18c59b10-15c6-4220-af92-6aa55d5755b1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.338855 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '447abd08211d49f9b503130ac5c380b7', 'request_id': 'req-18c59b10-15c6-4220-af92-6aa55d5755b1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.426533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '447abd08211d49f9b503130ac5c380b7', 'request_id': 'req-18c59b10-15c6-4220-af92-6aa55d5755b1'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.439610 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] The token cache is being invalidate because project 447abd08211d49f9b503130ac5c380b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:40:25.445955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.445955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.445955 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1450/2896] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:40:25 2022] DELETE /identity/v3/projects/447abd08211d49f9b503130ac5c380b7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 18:40:25.447243 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:25.448944 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:25.466197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b1aeb-5366-401c-8546-8261fad3dad5 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:25.539069 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:25.539997 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:25.540286 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:25.540607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] PATH_INFO: `/v3/users/ebdeaae563b543a4b07eb7b246b888ea` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:25.542020 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdeaae563b543a4b07eb7b246b888ea)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:25.544096 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:25.761692 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdeaae563b543a4b07eb7b246b888ea'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.762372 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdeaae563b543a4b07eb7b246b888ea'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.774517 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdeaae563b543a4b07eb7b246b888ea'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.781733 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdeaae563b543a4b07eb7b246b888ea'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.783424 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:40:25.855554 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdeaae563b543a4b07eb7b246b888ea'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:25.862508 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1447/2897] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:40:25 2022] DELETE /identity/v3/users/ebdeaae563b543a4b07eb7b246b888ea => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:40:26.143163 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:40:26.145713 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:40:26.161974 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c59b10-15c6-4220-af92-6aa55d5755b1 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:40:26.172104 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:40:26.231782 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:40:26.233407 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:40:26.233832 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:40:26.234258 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] PATH_INFO: `/v3/projects/e56d35c43e134bb7b29ea3e00d165cab` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:40:26.236387 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56d35c43e134bb7b29ea3e00d165cab)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:40:26.239124 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:40:26.453779 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56d35c43e134bb7b29ea3e00d165cab', 'request_id': 'req-e16b5d37-c915-4d16-995d-ddad53a05e85'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:26.460447 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56d35c43e134bb7b29ea3e00d165cab', 'request_id': 'req-e16b5d37-c915-4d16-995d-ddad53a05e85'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:26.557422 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56d35c43e134bb7b29ea3e00d165cab', 'request_id': 'req-e16b5d37-c915-4d16-995d-ddad53a05e85'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:26.569494 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] The token cache is being invalidate because project e56d35c43e134bb7b29ea3e00d165cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80576) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:40:26.569841 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:26.570477 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:40:26.572537 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1451/2898] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:40:26 2022] DELETE /identity/v3/projects/e56d35c43e134bb7b29ea3e00d165cab => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:40:35.934966 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:40:36.566162 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:41:27.019594 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:41:27.020996 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:41:27.085634 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9b540-cda1-4e9a-a8ca-8e938c68e447 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:41:27.141226 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:41:27.141887 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:41:27.142180 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:41:27.142464 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:41:27.144259 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:41:27.155124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:41:27.156719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:41:27.165560 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1448/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:41:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:41:37.161598 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=46 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:42:26.583230 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:42:26.585786 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:26.599063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5d37-c915-4d16-995d-ddad53a05e85 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:42:26.602818 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:42:26.603473 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:42:26.603767 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:42:26.604048 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:42:26.606005 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:26.616707 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:42:26.618190 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:42:26.627889 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1452/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:42:26.903435 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:42:26.905086 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:26.917251 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e086c-5838-4fc0-b760-c8bee0c8763a service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:42:26.922073 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:42:26.927040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:42:26.927040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:42:26.927040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:42:26.927040 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:26.937254 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:42:26.938788 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:42:26.951737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1449/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:42:27.053994 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:42:27.063078 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30918971-3951-47d8-b12c-55c6738e3dd9 service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:42:27.066918 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd67da2713214172989b5c689a8885f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:42:27.067575 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:42:27.067866 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:42:27.068153 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:42:27.070775 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:27.155623 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:42:27.157182 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:42:27.170555 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1453/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:42:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:42:27.238197 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:42:27.239615 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:27.321010 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f690-0406-4ce2-9228-f50d2e17df8b service neutron] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:42:27.326168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:42:27.326832 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:42:27.327118 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:42:27.327397 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:42:27.329225 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:27.342515 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:42:27.344123 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:42:27.355429 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1450/2903] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:42:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:42:28.248431 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:42:28.255880 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:28.265710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59ea9f-e498-46fa-93af-4d8444bcc08a service neutron] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:42:28.271328 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:42:28.272318 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:42:28.272615 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:42:28.272892 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:42:28.274661 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:42:28.287011 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:42:28.289503 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:42:28.300710 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1454/2904] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:42:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:42:37.350732 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=38 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:42:38.295882 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=46 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:43:59.295988 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:43:59.297865 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:43:59.411912 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a44559-5c5d-4cc8-9988-d3c22fa6ca19 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:43:59.417710 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:43:59.418377 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:43:59.418661 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:43:59.418960 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:43:59.420779 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:43:59.434168 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:43:59.435782 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:43:59.457108 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1451/2905] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:43:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:44:01.867540 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:01.940660 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91eb997-4fe9-4a9c-8a6a-243fb9541b7f service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:01.946563 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71160dfe2635497abc4cd1c1bffc4531', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:01.947276 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:01.947558 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:01.947844 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:01.949788 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:01.968526 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:01.970206 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:01.983790 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1455/2906] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:44:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:44:02.954550 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:02.956319 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:03.000301 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a52d2e-57b9-4180-b103-2247cfaac341 service nova] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:03.006171 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:03.006936 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:03.007230 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:03.007518 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:03.009399 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:03.022609 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:03.024229 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:03.035119 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1452/2907] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:44:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 18:44:08.499342 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:08.512259 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98b69c-3e77-4b1c-bf2c-5898777756c2 service nova] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:08.517511 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:08.518167 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:08.518451 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:08.518731 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:08.520674 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:08.532734 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:08.534708 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:08.544911 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1456/2908] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:44:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:44:12.313673 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:12.325126 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17492f-bba1-4647-8224-3719371cab41 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:12.330516 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:12.331183 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] REQUEST_METHOD: `GET` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:12.331457 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:12.331737 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:12.334166 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:12.346584 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:12.348233 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:12.359719 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1453/2909] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:44:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 18:44:12.458586 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:12.460280 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:12.529725 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeaec8-7b99-4c5a-8fc0-ab5a7d22e467 service placement] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:12.534221 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:12.535091 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:12.535376 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:12.535656 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] PATH_INFO: `/v3/users/11c8efca90fa4e309ada93f3eecda4cc` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:12.536919 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c8efca90fa4e309ada93f3eecda4cc)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:12.538868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:12.712413 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c8efca90fa4e309ada93f3eecda4cc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:12.719465 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c8efca90fa4e309ada93f3eecda4cc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:12.731339 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c8efca90fa4e309ada93f3eecda4cc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:12.732160 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c8efca90fa4e309ada93f3eecda4cc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:12.732677 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c8efca90fa4e309ada93f3eecda4cc'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:12.799584 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:44:12.804560 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1457/2910] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:44:12 2022] DELETE /identity/v3/users/11c8efca90fa4e309ada93f3eecda4cc => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:44:13.087527 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:13.099124 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a8525-c9bb-40e2-b17b-82728614bce0 service placement] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:13.159234 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:13.160110 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:13.160431 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:13.160717 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] PATH_INFO: `/v3/projects/cbb64781270b435aaf0eb5e2c4d290b0` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:13.162524 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb64781270b435aaf0eb5e2c4d290b0)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:13.164500 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:13.254179 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:44:13.271883 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb64781270b435aaf0eb5e2c4d290b0', 'request_id': 'req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.278099 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb64781270b435aaf0eb5e2c4d290b0', 'request_id': 'req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.348052 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb64781270b435aaf0eb5e2c4d290b0', 'request_id': 'req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.359712 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] The token cache is being invalidate because project cbb64781270b435aaf0eb5e2c4d290b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:44:13.360006 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.360685 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.362283 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1454/2911] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:44:13 2022] DELETE /identity/v3/projects/cbb64781270b435aaf0eb5e2c4d290b0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:44:13.367274 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:13.368805 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:13.382534 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaed426-32d1-4c20-a3ba-b622a01be3ba admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:13.439166 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:13.440042 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:13.440365 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:13.440643 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] PATH_INFO: `/v3/users/fb822c5499924685864f4d506bebf381` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:13.441868 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb822c5499924685864f4d506bebf381)` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:13.444039 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:13.641798 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb822c5499924685864f4d506bebf381'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.649587 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb822c5499924685864f4d506bebf381'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.662356 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb822c5499924685864f4d506bebf381'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.663099 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb822c5499924685864f4d506bebf381'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.663714 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.notifications [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb822c5499924685864f4d506bebf381'} {{(pid=80576) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:13.732063 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1458/2912] 10.222.0.47 () {58 vars in 1223 bytes} [Tue Jul 12 18:44:13 2022] DELETE /identity/v3/users/fb822c5499924685864f4d506bebf381 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:44:14.007541 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Authenticating user token {{(pid=80577) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:44:14.009029 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.fernet_utils [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80577) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:44:14.026426 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afef71a-f0bf-4c71-bf8f-67f88a5534d7 admin admin] Validating token access rules against request {{(pid=80577) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:44:14.100192 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26ab52b942da412ea520d26ff61916e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cecdefe3d592442ba46417c3dd5f54f9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80577) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:44:14.101139 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:44:14.101432 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:44:14.101721 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] PATH_INFO: `/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484` {{(pid=80577) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:44:14.103212 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdb32f2e54e4cfe8a8d2441f0ac4484)` {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:44:14.105365 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] RBAC: Authorization granted {{(pid=80577) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:44:14.219578 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdb32f2e54e4cfe8a8d2441f0ac4484', 'request_id': 'req-5f7da976-de91-470d-9a88-52bd38bd53c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:14.226341 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdb32f2e54e4cfe8a8d2441f0ac4484', 'request_id': 'req-5f7da976-de91-470d-9a88-52bd38bd53c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:14.325652 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdb32f2e54e4cfe8a8d2441f0ac4484', 'request_id': 'req-5f7da976-de91-470d-9a88-52bd38bd53c0'} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:14.337653 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] The token cache is being invalidate because project 4cdb32f2e54e4cfe8a8d2441f0ac4484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80577) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 18:44:14.337969 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:14.338607 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG keystone.notifications [None req-5f7da976-de91-470d-9a88-52bd38bd53c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80577) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 18:44:14.340152 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: [pid: 80577|app: 0|req: 1455/2913] 10.222.0.47 () {58 vars in 1232 bytes} [Tue Jul 12 18:44:14 2022] DELETE /identity/v3/projects/4cdb32f2e54e4cfe8a8d2441f0ac4484 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 18:44:23.728145 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG dbcounter [-] [80576] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80576) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:44:24.333616 dv-cinder-844881-1-iscsi devstack@keystone.service[80577]: DEBUG dbcounter [-] [80577] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80577) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 18:49:27.270090 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Authenticating user token {{(pid=80576) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 18:49:27.271655 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:49:27.351533 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a674fa-fece-48c1-91b8-ee8fba2c9f42 admin admin] Validating token access rules against request {{(pid=80576) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 18:49:27.421424 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096d7e0380e84c97a9b72b4203a19abd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd09adfd8fd6e4533ba29be1b6222468b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80576) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 18:49:27.422141 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] REQUEST_METHOD: `GET` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 18:49:27.422434 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] SCRIPT_NAME: `/identity` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 18:49:27.422717 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80576) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 18:49:27.429177 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.fernet_utils [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80576) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 18:49:27.443681 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 18:49:27.445607 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a61ab-5dd5-4587-a727-d11d8ef25123 service placement] RBAC: Authorization granted {{(pid=80576) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 18:49:27.458499 dv-cinder-844881-1-iscsi devstack@keystone.service[80576]: [pid: 80576|app: 0|req: 1459/2914] 10.222.0.47 () {60 vars in 1365 bytes} [Tue Jul 12 18:49:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)